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CHAPTER  I 


INTRODUCTION 


1.  S 


Surnrna 

Consider  a  system  which  is  composed  of  some  number  of  subunits 
or  components.  The  structure  of  the  system  is  the  relationship  between 
the  functioning  of  various  components  and  that  of  the  whole  system. 
Structure  can  be  specified  completely  in  terms  of  the  minimum  cut  sets 
These  are  sets  of  components  for  which: 
the  system  can't  work  if  they  are  removed  or  failed, 
if  any  component  in  the  min  cut  set  is  restored,  then  the 
system  will  work. 


A 


Example  1.1  (parallel  system  of  2  components) :  The  system 
works  ♦  at  least  one  component  is  working;  min  cut  set  »  (1,  2}. 


Example  1.2  (series  system  of  3  components) :  The  system  works 
all  components  work;  min  cut  sets  =  {1},  {2},  {3}.  (2,  3}  is  not  a 
min  cut  set  because  even  though  the  system  is  failed  when  2  and  3  are 
failed,  restoring  component  2  or  3  does  not  restore  the  system. 

* 

Intuitively  speaking,  a  system  is  coherent  if: 

(i)  Suppose  it  is  down  when  a  certain  subset  of  components 
is  failed;  it  will  not  start  working  if  even  more 
components  fail. 

*See  Ref.  [2],  Chapter  1,  for  precise  mathematical  definition. 
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(ii)  If  It  Is  working  when  a  certain  subset  of  components 
is  working,  it  will  not  fail  if  even  more  components 
are  restored  to  working  order. 

A  Coherent  System  Repair  Model  models  the  maintenance  over  an 
infinite  continuous  time  horizon  of  a  stochastically  deteriorating 
finite  set  of  components  which  form  a  coherent  system.  There  are 
costs  for  repair  and  system  failure  and  the  lifetime  of  each  component 
is  a  random  variable  with  known  expected  value.  The  objective  is  to, 
assuming  decisions  can  be  made  at  the  instant  of  a  component  failure, 
minimize  the  long  run  expected  cost  per  unit  time  or  the  total 
expected  cost,  whichever  is  desired.  Decisions  can  be  made  to  do 
nothing  or  to  repair  some  subset  of  the  failed  components;  when  the 
system  is  down,  the  do-nothing  option  is  eliminated  except  in  cases 
of  noninstantaneous  repair.  A  more  detailed  discussion  of  the 
parameters  which  can  be  varied  in  these  models  will  appear  later  in 
the  chapter. 

The  purpose  of  this  thesis  is  to  formulate  and  describe 
different  types  of  coherent  system  repair  models  and  then  gather  as 
much  and  as  general  information  as  possible  on  the  nature  and  form 
of  optimal  policies  involved.  In  the  case  where  system  components 
have  exponential  (not  necessarily  identical)  lifetime  distributions, 
the  model  can  be  formulated  as  a  continuous  time,  infinite  horizon 
Markov  decision  chain  with  no  discounting  and  a  finite  state  and 
decision  space.  States,  which  are  nontlme-dependent ,  depend  on  the 
states  of  the  individual  components:  working,  failed,  or  under 


repair.  The  bulk  of  Information  will  come  for  models  with 
exponential  components,  as  Markov  chain  theory  is  a  very  powerful 
tool.  If  the  components  are  nonexponential  (and  non-Erlang)  the 
process  as  defined  above  will  be  non-Markov.  In  that  case.  It  Is 
no  longer  sufficient  to  have  nontlme-dependent  states.  The  problem 
is  much  more  difficult  and  will  be  beyond  the  scope  of  this  thesis. 

After  initial  general  discussion,  three  types  of  models  will 
be  looked  at  in  detail,  all  requiring  exponential  or  Erlang 
components.  The  Basic  Model  (Chapters  II  &  III)  allows  exponential 
components  to  be  working  or  failed  (on/off).  Repair  time  is  assumed 
instantaneous  with  unlimited  service  facilities.  The  Degradation 
Model  (Chapter  IV)  has  the  same  component  lifetime  and  repair 
assumptions  as  the  Basic  Model  except  that  here  each  component  can 
be  in  any  one  of  a  finite  number  of  observable  degradation  states, 
with  the  first  state  being  "new"  and  the  last  state  being  "failed". 
Following,  the  Noninstantaneous  Repair  Model  (Chapter  V)  takes  the 
Basic  Model  and  allows  for  exponential  repair  times  (noninstantaneous) . 

Results  obtainable  for  each  of  the  three  models,  using 
Veinott's  solution  technique  for  Markov  decision  chains 
[see  Section  l.A]  (i.e.,  finding  the  long  run  expected  cost/time  for 
each  policy  and  then  among  those  policies  for  which  this  is  minimized, 
find  the  total  expected  cost,  etc.)  are  Included  in  the  model 
Chapters  II  through  V.  In  each  case,  the  specific  structure  of  the 
underlying  Markov  chain  allows  us  to  get  conditions  which  eliminate 
certain  types  of  policies  from  being  optimal.  For  some  cases,  notably 


* 


those  for  which  components  are  identical,  a  complete  general  optimal 
solution  is  obtained. 

Chapter  VI  looks  at  the  theory  and  algorithms  available  for 
getting  computational  results,  and  implements  a  linear  programming 
method.  This  provides  a  look  at  optimal  solutions  in  cases  where 
policy  enumeration  by  hand  is  too  messy  or  none  of  the  theorems  in 
Chapters  II-V  apply.  Some  test  problem  results  are  presented. 

In  Chapter  VII,  results  from  the  previous  five  chapters  are 
compiled  and  compared.  Basic  policy  forms  and  restrictions  which 
interrelate  the  models  are  presented,  and  conclusions  about  how 
varying  model  parameters  affect  the  optimal  solution  are  drawn.  Then 
some  of  the  possible  applications  of  these  models  or  variations  on 
them  are  discussed.  Extensions  and  topics  for  future  research 
conclude  the  chapter  and  thesis. 


2.  A  Survey  of  Maintenance  Model  Literature 

A  maintenance  model  models  the  control  and  surveillance  of  a 
stochastically  deteriorating  system.  For  two  decades  there  has  been 
a  large  and  continuing  interest  in  such  models  and  the  number  of 
relevant  papers  in  the  literature  reflects  it.  Figure  1.1  below  gives 
a  classification  of  maintenance  models  according  to  the  type  of 
maintenance  problem  modeled: 
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FIGURE  1.1  Ref. [29] 
Classification  of  Maintenance  Models 


Discrete  Time  Continuous  Time 


1. 

Complete  Information 

1. 

Control  Theory 

2. 

Incomplete  Information 

2. 

Age  Dependent  Replace¬ 

ment  Models 

3. 

Maintenance  and 

Inventory 

3. 

Shock  Models 

* 

4. 

Interacting  Repair 

Activities  Models 

5. 

Incomplete  Information 

Further  broken  down  in  Figure  1.2. 


Discrete  time  models  select  actions  at  discrete  points  in 
time.  These  models,  as  appear  in  the  literature,  utilize  information 
regarding  the  degree  of  deterioration  of  the  unit  or  units  in  order 
to  select  the  best  repair  or  maintenance  action  at  certain  discrete 
points  in  time.  In  some  cases,  an  inspection  must  be  made  to 
ascertain  the  state  of  the  system  before  repair  decisions  are  made, 
while  in  others  it  is  assumed  the  current  state  of  the  system  is 
always  known.  For  incomplete  information  models,  actions  must  be 
taken  under  uncertainties  about  costs,  underlying  failure  laws,  or 
observations  of  state.  Inventory  models  involve  decisions  concerning 
periodic  restocking  of  inventories  of  spare  parts.  Note  that  in  all 
these  models,  the  system  is  treated  as  a  unit  in  terms  of  formulation. 
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Che  individual  components  being  ignored.  Most  formulations  for 


| 


discrete  time  models  are  based  on  Markov  decision  theory  or  inventory 
theory,  thus,  linear  and  dynamic  programming  are  primary  solution 
techniques.  See  [21],  [26],  [33]  for  details  and  specific  models. 

Continuous  time  models  do  not  restrict  maintenance  or 
inspection  activity  to  a  particular  set  of  discrete  points  in  time. 

Control  theory  models  permit  maintenance  activity  to  occur  as 
a  continuous  stream.  The  decision  maker  must  optimize  over  functions 
m(0  where  m(t)  is  the  maintenance  expenditure  rate  at  time  t. 

Age  Dependent  Replacement  Models  allow  maintenance  only  at 
certain  discrete  points  in  time,  e.g. ,  replacing  an  item  when  it 
reaches  a  certain  age.  This  problem  is  just  the  continuous  time 
analog  of  the  discrete  time  one  with  deterioration  stages.  Different 
models  vary  cost  assumptions,  types  of  repairs  allowed  and  numbers 
of  spares  around.  Again,  the  system  is  treated  as  a  unit. 

Shock  Models  regard  the  unit  as  subject  to  exterior  shocks, 
each  of  which  damages  (causes  wear)  in  such  a  way  that  the  damage 
accumulation  up  to  a  particular  time  defines  the  unit's  probability 
of  failure  of  that  time.  This  assumption  differs  from  the  standard 
assumption  that  the  time-to-failure  random  variable  of  a  unit  is 
intrinsic  to  that  unit. 

The  Coherent  System  Repair  Model  falls  into  the  category  of 
interacting  repair  activities  models,  the  only  ones  where  maintenance 
policies  exploit  interactions  among  the  units  of  a  system.  Figure 
1.2  presents  ways  in  which  a  maintenance  policy  achieves  this: 


FIGURE  1.2 


Types  of  Maintenance  Policies  Which  Exploit  System  Structure 


* 

1.  Opportunistic 

(economies  of  scale) 


how  many?  (cheaper 
to  repair  more  at  once) 


2.  Cannibalization 

3.  Multi-stage  Repair 


units  of  same 
type  utilized 
at  different 
locations  In 
system 


no  new  Items  In  system 


new  Items  can  enter 
system 


4.  Variable  Rate 
Repair 


repair  capacity  limited  and  under 
decision  maker's  control 


5.  Which  ones? 


(if  system  has  nonidentical  components 
with  several  Items  failed) 


6.  Repair  or  not? 


(depending  on  how  much  of  the  system 
is  still  operative) 


type  of  decisions  allowed  in  a  coherent  system  repair  model. 


Research  on  problems  involving  cannibalization,  multi-stage 
repair,  variable  rate  repair,  and  opportunistic  policies  has  been 
going  on  for  some  10-15  years,  (see  [33],  Reference  list).  "Repair  or 
not"  type  policies  for  deteriorating  units  were  among  the  first  to  be 
studied,  the  best  example  being  a  "control  limit  policy"  which  says 
"repair  or  replace  the  unit  when  it  reaches  a  certain  level  of 
deterioration". 

However,  until  just  recently,  no  work  has  been  done  on  optimal 
repair  order,  i.e.,  choosing  which  item  to  repair  if  several  components 
are  down,  or  on  models  which  can  treat  the  type  of  system  as  a  variable. 


It  was  a  desire  to  get  some  results  in  these  areas  that  motivated 
the  Coherent  System  Repair  Model  formulation.  It  models  a 
deteriorating  system  of  components  as  do  other  interacting  repair 
activities  models  but,  unlike  the  others,  the  state  of  the  components 
explicitly  determines  the  state  of  the  system.  Varying  the  component 
configurations  for  which  a  "penalty"  is  incurred  for  system  failure 
allows  introduction  of  the  system  type  as  a  variable. 

Two  recent  works  on  optimal  repair  order  have  been  by  Derman, 
Lieberman,  and  Ross  (1978)  [15]  and  D.  R.  Smith  (1978)  [29].  Derman, 
Lieberman,  and  Ross  consider  the  same  type  problem  as  a  coherent 
system  repair  model  with  exponential  components,  using  an  N-server 
queuing  system  to  show  that  the  policy  which  always  repairs  the  failed 
component  whose  failure  rate  is  smallest  stochastically  maximizes  the 
number  of  working  components.  However: 

(a)  decisions  to  not  repair  even  though  the  server  is 
free  are  not  allowed 

(b)  decisions  to  repair  more  than  one  unit  at  a  time  are 
not  allowed,  (single  repairman) 

(c)  different  objective  function  -  no  cost  structure 
(maximize  number  of  working  components) 

Smith  considers  a  series  system  of  n  independent  components 
as  an  irreducible  continuous  time  Markov  chain  and  gets  the  same 
result  as  Derman,  et  al.  However: 

(a)  he  has  no  cost  structure,  his  objective  is  minimizing 
the  long  run  fraction  of  time  the  system  is  up 


(b)  he  does  allow  for  "do  nothing"  options  but  only  a 
single  server 

(c)  he  solves  using  a  different  technique 

(d)  results  apply  to  a  series  system  only. 

In  a  coherent  system  repair  model,  time  is  not  a  variable  - 
the  level  of  deterioration  of  the  system  is  measured  by  which  and 
how  many  of  the  components  are  working  or  are  in  a  certain  level 
of  degradation.  In  general,  with  complicated  systems  having 
different  components,  one  would  expect  some  very  complicated 
optimal  policies  which  would  be  impossible  to  guess  but  well  worth 
finding.  When  the  components  are  identical,  things  simplify 
considerably  for  k-of-n  systems,  as  the  number  of  components  up 
is  now  the  only  determining  factor  in  system  deterioration  and 
failure. 

Although,  in  the  most  general  cases,  one  cannot  hope  for 
exact  solutions,  except  by  computer,  one  can  and  does  look  for 
restrictions  on  the  very  large  initial  decision  space  (can  repair 
any  subset  of  failed  components  at  the  instant  of  any  component 
failure).  In  the  course  of  investigation  of  various  models,  two 
such  restriction  types  will  appear: 

(1)  never  repair  more  than  a  certain  number  of  components 
at  a  time  in  certain  states 

(2)  never  repair  when  more  than  a  certain  number  of 
components  are  working. 

Both  these  results,  besides  being  useful  from  a  system  operator's 
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viewpoint,  are  useful  in  lowering  the  computation  time  needed 
to  find  an  optimal  policy  in  cases  where  it  cannot  be  done  by 
hand. 

Initially,  maintenance/inspection  models  possessed  an  elegance 
and  a  simplicity  which  led  to  easily  implementable  results  and  exact 
optimal  policies.  More  recent  results  have  become  increasingly 
complex,  requiring  a  large  computer  to  implement  policies.  Coherent 
system  repair  model  results  are  some  of  each  -  in  general  cases  it 
is  necessary  to  use  the  computer  or  state  theorems  which  eliminate 
certain  policy  types  but  in  many  specific  cases,  exact  "optimal" 
policies  can  and  have  been  worked  out,  especially  in  cases  where 
components  are  identical.  The  formulation  is  simple  and  makes  for 
easy  usage  of  the  model  under  a  great  variety  of  hypotheses. 

The  following  section  gives  a  description  of  coherent  system 
repair  models,  their  basic  features  and  parameters. 


3.  Description  of  Model 

The  formulation  of  a  coherent  system  repair  model  is  quite 
general.  Figure  1.3  summarizes  the  features  of  such  a  model.  As  is 
easily  seen,  there  are  a  lot  of  unspecified  parameters  which  have 
to  be  defined  before  using  the  model.  These  include:  the  objective, 
type  of  system,  distribution  of  component  lifetimes,  costs,  and 
several  others  described  in  Figure  1.4. 
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FIGURE  1.3 


Features  of  a  Coherent  System  Repair  Model 


(1)  Models  the  maintenance  of  a  stochastically  deteriorating 
system  of  n  independent  components 

(2)  System  states  (and,  thus,  decisions)  are  functions  of 
component  states  but  not  functions  of  time  (due  mainly 
to  exponential  components) 

(3)  Cost  structure  for  repair  or  system  failure 

(4)  Formulation  as  a  continuous  time  Markov  decision  chain, 
infinite  horizon,  no  discounting  (finite  state  and 
action  space) 

(5)  Large  decision  space:  at  each  state,  can  decide 

a.  repair  or  not  (must  have  some  repair  action  going 
if  system  is  down) 

b.  if  so,  which  subset  of  failed  components  to  repair 

(6)  Decision  times  are  limited  to  times  of  a  component 
failure  and  are  made  immediately  following  such 

(7)  Some  resulting  policy  limitations: 

a.  don't  repair  unless  a  certain  number  of  components 
have  failed 

b.  don't  repair  more  than  a  certain  number  of  components 
at  one  time 

(8)  Can  get  exact  "optimal"  policies  in  some  cases, 
especially  when  components  are  identical 

(9)  Objective:  minimize  V  (long  run  expected  cost  per 
unit  time)  or  (total  expected  cost) 

Start  with  all  components  up 

(10)  Solution  techniques:  use  structure  of  underlying  Markov 
chain  to  get  conditions  on  V_^,  Vq. 
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FIGURE  1.4 


parameter 
1.  Objective: 


2.  System  Type: 


3.  Component  Lifetime 
Distributions: 


4.  States: 


5.  Repair: 


6.  Costs: 

in  all  models 

to  fix  unit  i 

p  per  system 
breakdown 


Model  Parameters 


possible  values 

-  min  V  the  long  run  expected 
cost  per  unit  time 

-  min  Vq,  the  total  expected  cost 


-  general  coherent 

,  c  =  1  parallel 

-  k-of-n^  ,  . 

■*  k  =  n  series 


-  exponential 


identical  components 


-  nonidentical  components 
-  gamma,  integer  shape  parameter 


-  components  on/off  (which) 

-  components  in  degradation  states 
(which) 

-  components  on/off  or  in  service 
(which) 

-  number  of  components  in  various 
states  (k-of-n  system,  identical 
components  only) 


-  instantaneous 

-  exponential  (s  servers) 

-  Erlang,  integer  shape  parameter 

plus  -  fixed  charge  L,  per  repair  decision 

-  no  fixed  charge  L 

-  labor  cost  £./server/unit  time 
(noninstantaneous  repair  only) 
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The  evo  objective  functions  used  are  among  those  proposed 
by  Velnott  [30,  31]  in  his  method  for  solving  continuous  time  Markov 
decision  chains  with  infinite  time  horizon  and  no  discounting.  See 
Section  1.4,  [31]  or  [11]  for  a  brief  presentation  of  the  theory 
behind  it. 

Changing  parameters  which  affect  the  number  and  type  of 
states  in  the  Markov  chain  cause  such  overwhelming  changes  in  the 
model  that  each  is  studied  separately  as  a  "model  type".  Such 
parameters  include  the  component  lifetime  distributions,  types  of 
component  states  existing,  and  the  repair  option.  The  following 
three  model  types  were  selected,  a  chapter  being  devoted  to  each. 

(I)  Basic  Model  comp,  life  *  exponential 

states  *  on/off 

repair  =  instantaneous 

(II)  Degradation  comp,  life  *  exponential  ^ 

Model  states  *  degradation  pChapter  "1 

levels  |_  IV  J 

repair  *  instantaneous 

(III)  Noninstantaneous  comp,  life  =  exponential 

Repair  Model  states  =  on/off,  in  [Chapter 

service  (_  V 

repair  =  exponential/ 

Erlang 

Changing  system  type  or  costs  affects  only  the  cost  structure  and 
such  variations  are  studied  within  each  model  type. 

Obviously,  other  combinations  of  parameters  could  be  used  to 
get  other,  more  complicated,  model  types.  However,  these  three  are 
the  simplest,  give  the  most  results,  and  give  a  good  feeling  for 


Chapters! 
II,  III  | 


the  types  of  optimal  policies  being  dealt  with.  Given  any  other 
combination  with  exponential  or  Erlang  repair  and  component  lifetimes, 
an  optimal  solution  can  be  sought  using  computer  algorithms  like 
those  in  Chapter  VI  if  the  number  of  states  is  not  too  large. 

4.  Discussion  of  Optimality  Criteria 

In  this  section,  some  optimality  criteria  are  defined.  Which 
ones  will  be  used  in  this  thesis  and  why,  is  then  discussed. 

Suppose  we  are  given  a  continuous  time,  infinite  horizon 
Markov  renewal  programming  model  with  finite  state  and  action  space 
and  no  discounting.  It  is  sufficient  to  consider  only  nonrandomized 
policies,  (see  [11]).  The  case  of  exponential  transition  times 
simplifies  to  the  Markov  decision  chain  which  will  be  used  to 
represent  a  coherent  system  repair  model.  In  this  case  the  decisions 
or  the  holding  time  distributions  in  a  state  depend  only  on  the 
current  state,  not  on  the  next  state  as  for  the  general  Markov  renewal 
programming  model.  Also,  only  stationary  policies  need  be  considered 
(see  [8],  Example  4). 

The  problem  being  infinite  horizon,  the  objective  is  to 

minimize  the  expectation  of  undiscounted  cost  incurred  over  all  time, 

i.e.,  minimize  lim  V^(t)  over  y  where  V^(t)  is  the  expectation 
t-w*>  1  1 

of  undiscounted  cost  incurred  during  interval  [0,  t]  under  policy 
y  if  the  system  starts  in  state  i  ■  (12  •••  n)  ■  all  components 
working . 


Y 

UnforCunately,  11a  V'(t)  ■  •  in  most  cases  and,  thus.  Is 
t-*»  1 

not  a  good  quantity  to  compare  policies  with.  To  render  things 
finite  if  V(t)  grows  slower  than  exponentially,  a  positive 
"interest"  or  "discount*  rate  "s"  can  be  introduced  into  the 
model. 

CD 

Let  v^(s)  ■  /  e~8tdV^(t).  (taking  Laplace- 

0-  Stieltjes  transforms) 

6  v 

Definition;  6  is  s-optimal  if  v  (s)  _>  vT(s)  V  y  e  A, 

A  •  policy  space. 

Definition:  (Blackwell)  6  is  optimal  if  it  is  s-optimal 

V  sufficiently  small  s.  (AQpt  *  set  optimal  policies,  possibly 
empty) . 

As  the  behavior  of  v(s)  as  s  -*•  0+  is  intimately  related  to  that 
of  V(t)  as  t  -*■  ®,  we  use  Blackwell's  criterion  as  the  definition 
of  optimality. 

Given  a  definition  of  optimality,  it  is  now  necessary  to  show 
how  to  compute  on  optimal  policy  given  a  specific  Markov  decision 
chain  (or  renewal  program).  The  next  theorem  will  demonstrate  an 
approach  towards  doing  so. 


A  Markov  renewal  program  is  defined  by  the: 


(i)  states  1,  2,  N 

(ii)  decisions  ”  set  of  possible  decisions  in  state  i 

Vr 

(iil)  transition  probabilities  (x) , 

Q±j (X>  *  P{sn+1  ’  j’  Cn+1  ±  *  +  XlSn  "  i*tn  “  C’  6(i)  “  k} 
k 

(iv)  cost  structure  R^(x)  ■  expected  cost  incurred  during 
time  interval  [0,  min  (x,  t)],  Sq  ■  i,  6(i)  *  k. 

This  is  assumed  time  invariant. 

In  the  case  of  exponential  transition  probabilities,  i.e.,  the 
Markov  decision  process. 


<»> 


k 

r  k  1 

-Xtx 

_k ,  .  k 

,k 

-X  x 

-pij 

1-e 

;  R^x)  -  r± 

1-e 

Definition:  Normalized  moments 


(L  *  /  tndQ(t)/n! 
0- 


R  -  /  tndR(t)/n! 
n  0- 


N  x  N  matrix 


N  x  l  vector 


Note:  Qq  *  transition  matrix  of  embedded  Markov  chain. 

Note:  Well  known  Markov  chain  theory  says  as  n  -*■  «®, 

n  * 

(if  only  one  ergodic  class  present)  -►  P  ,  the  stationary 
transition  probability  matrix  which  can  be  computed  using  the 
equations 
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P  «-Q0> 


and 


Theorem:  Suppose  and  ®n+i  *8  defined 

and  finite.  Then  (eliminating  6,  1  from  notation  to  simplify) 
v(s)  ■  s  ^  +  Vq  +  sV^  +  •••  +  a°Vn  +  o(sn).  Moreover, 

Vi*  -1,  0,  n,  the  vector  Is  the  unique  solution  of 

equations: 

(I-Q0)Vi  -  b±  and  p\vi  "  p\ 

where 


"-1 


V  b-l 


0,  b. 


ci-l  '  Q1V1-1 


and 


.  ,i  1+2  . 

vi>0,  Cl*(-l)  \+1  +  C-DVi+i-j 


Proof :  see  [ll]. 


This  suggests  that  when  s  is  sufficiently  small,  the  decision 

maker  can  go  about  finding  an  optimal  policy  by  first  selecting  policies 

6  6 

6  so  as  to  minimize  V  break  ties  by  minimizing  V^,  etc.,  until 
a  unique  policy  is  arrived  at,  if  one  exists.  In  the  exponential  case, 
it  does  (see  [31]).  In  addition,  there  it  can  be  shown  that  “  A*, 

where  A_ ^  ■  A,  4^  *  {{  e  Ak-l^Vk  —  Vk  *  ®  ^  Ak-1^*  an<* 

A  *  lim  A.  . 

■  k-"»  k 
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and  are  Chose  for  which  the  long  run  average  cost  per  unit  time  is 
minimized . 


Definition:  optimal  policies  are  called  bias  optimal  and 

are  those  for  which  the  total  expected  cost  is  minimized.  In 
particular  VQ  solves  (I-Q0)Vq  -  Ro“^iV-i  and  P  ^iV0  “  P  ^“Ri'K^2V-1^ ’ 

In  solving  specific  coherent  system  repair  models  in 
successive  chapters,  V  ^  and  will  be  used  as  optimality  criteria. 

With  other  past  maintenance  models  which  use  a  Markov  decision  chain 
structure  with  costs,  the  objective  was  almost  always  to  minimize 
V  the  average  cost  per  unit  time.  An  example  will  be  given  that 
demonstrates  the  need  for  going  to  if  it  is  desired  to  find  the 

optimal  decisions  for  the  transient  states. 

Example  1.3  (parallel  system,  n  ■  2  components,  no  fixed 
charge, states  ■  which  components  up  -  {12,  1,  2,  0}): 


decisions:  A12  *  {A}  A.  *  {A,  R?}  A„  -  (A,  R  } 


cost: 


R1 

r0  "  ?  +  K1 
R2 

r0  “P  +  K2 
R12 

r0  =  P  +  Kx  +  K2  . 


It  turns  out  that  if  p  is  small  enough,  in  0  is 

expected  cost 

V optimal  if  component  1  has  a  smaller  e^ecte~<j  Yife  rat^°  than 
component  2.  The  V  ^  optimal  policy  does  not  specify  what  to 

do  in  states  1  or  2  in  this  case. 


i 


Basically,  given  a  policy  6,  an  underlying  Markov  chain 
structure  is  formed  which  contains  some  ergodic  states  and  some 
transient  ones.  Finding  a  V_^  optimal  policy  locates  that 
ergodic  chain  of  states  which  minimizes  long  run  average  cost/unit 
time  over  all  other  policies  which  have  a  different  ergodic 
structure.  However,  except  for  a  few  special  cases,  there  will 
be  many  V optimal  policies  i.e.,  when  one  can  change  any  decision 
in  the  transient  states  without  forming  other  ergodic  states,  V_^ 
will  be  unchanged.  Thus,  to  find  out  the  optimal  thing  to  do  in 
transient  states  before  reaching  the  ergodic  chain,  from  which  one 
never  leaves,  it  is  necessary  to  look  at  Vq. 

The  next  question  is  why  then,  given  the  above  argument,  not 
keep  going  with  V^,  ...  etc.,  until  a  unique  policy  is  found? 
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Besides  being  computationally  messy,  it  is  unnecessary.  Any  ties 
with  Vq  can  be  broken  by  changing  one  of  the  costs  or  mean 
component  lifetimes  by  some  e,  however  small.  Any  data  measured 
to  use  in  such  a  model  is  only  going  to  be  precise  to  a  certain 
decimal  point  so  one  can  always  change  the  insignificant  portion 
around  to  break  ties  or  just  leave  them  and  call  them  equivalent 
policies. 

The  fact  that  in  a  coherent  system  repair  model,  the  states 
and  costs  are  nontime-dependent  with  no  discounting,  enables  us  to 
specify  an  "optimal"  decision  in  every  state  by  only  considering 
V_^,  Vg.  The  Vq  criterion  places  equal  weight  on  costs  at 
different  times,  which  suffices  to  determine  an  optimal  policy 
given  model  assumptions  to  minimize  total  expected  cost,  i.e., 
there  is  no  advantage  given  to  a  policy  which  has  the  same  total 
costs  as  another  but  less  of  the  cost  is  likely  to  be  incurred 
earlier,  thus,  allowing  for  more  cash-on-hand  for  a  while,  even 
though  the  total  spent  will  end  up  the  same  (type  of  policies 
distinguishes).  For  further  economic  interpretations  of 
v_l*  VQ,  V  see  Veinott  [31]. 
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CHAPTER  II 


THE  BASIC  MODEL 


1.  Description  of  Model 

The  Basic  Model  is  the  simplest  of  the  coherent  system  repair 
models,  but  by  no  means  uninteresting.  In  it,  assumptions  are  as 
follows  regarding  fixing  model  parameters: 

States :  components  are  on  or  off,  either 

(a)  depending  on  which  components  up  if  system 
j4  k-of-n  or  different  components 

(b)  depending  on  number  of  components  up  if 
system  =  k-of-n  and  same  components. 

Repair :  instantaneous,  unlimited  service. 

Component  Lifetimes:  exponential,  nonidentical  components 

order  the  components  so  that  number  1  has  smallest 
mean  lifetime  (or  largest  parameter),  i.e., 

-v 

L^  3  lifetime  of  number  i  ~  1  -  e  =  p{L^  <_  t) 

where  A,  >  A„  >  • • •  >  A  . 

12  n 

Variables  within  the  Basic  Model  structure  are  the  objective 
(V or  VQ),  the  type  of  system,  and  the  existence  or  not  of  a 
fixed  charge  as  well  as  costs  to  repair  a  component,  penalty  cost 
for  system  failure,  and  the  mean  lifetimes  of  the  various  components 
Section  2  gives  results  for  the  case  of  no  fixed  charge,  V  optimal 


In  Section  3,  what  happens  for  positive  fixed  charge  is  looked  at. 


It  should  be  noted  that  the  type  of  system  can  be  anything 
but  series.  Due  to  the  instantaneous  repair,  the  requirement  that 
repair  be  undertaken  on  some  component  when  the  system  fails  and 
the  fact  that  the  decision  space  is  to  repair  some  subset  of  the 
failed  components,  there  are  no  decisions  to  be  made  in  the  series 
case.  The  only  possible  states  are  {all  components  up}  or 
{i  down},  some  1  £  i  <  n.  In  the  former,  no  repair  is  done  since 
all  components  are  up  and  in  the  latter,  component  i  is  repaired 
automatically.  One  could  drop  the  requirement  of  repair  when  the 
system  breaks  down,  thus,  allowing  more  states  and  decisions  but, 
besides  unnecessarily  complicating  the  problem,  it  would  be 
unrealistic  to  assume  that  a  decision  maker  would  want  to  do  nothing 
if  the  whole  system  he  is  in  charge  of  breaks  down,  (if  the  system 
is  coherent). 

The  fixed  charge  is  denoted  L,  and  is  >_  0.  It  costs 
>  0  to  repair  component  i  (an  instantaneous  lump  sum  since 
instantaneous  repair)  and  p  >  0  is  charged  each  time  the  system 
fails. 

Component  lifetimes  being  exponential,  the  model  can  be 
formulated  as  a  continuous  time  Markov  decision  chain  with  infinite 
time  horizon  as  mentioned  in  Chapter  I.  To  define  the  model 
completely,  we  need: 
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(1)  State  space  (states  depend  on  which  components  up) 

(2)  Decision  space  (decisions  available  in  each  state) 

(3)  Transition  structure  (probabilities  of  changing  states) 

(4)  Cost  structure  (cost  of  decisions  in  each  state) 

(5)  Objective  function 

The  following  notation  is  now  introduced  and  will  be  used 
throughout . 


Figure  2.1 

Coherent  System  Repair  Model  -  Notation 


n 


s 


6 


A 


Ft(t) 


V 


i 


number  of  components 

a  state,  s  C  (l,  nl  =  {all  components 

working) 

a  policy  involving  decisions  in  each  state  s 
(to  repair  or  not  and,  if  so,  which  ones  and 
how  many) 

decision  to  do  nothing  (can  be  made  in  any 
state  in  which  system  is  not  failed) 

decision  to  repair  the  set  ftg  of  components 

(can  be  made  in  any  state  s  where  all  elements 

of  ft  are  failed) 
s 

matrix  of  transition  probabilities  between 
various  states  (depends  on  policy  6  and 
lifetime  distributions  of  components) 

lifetime  distribution  of  component  i 

*v 

=  1  -  e  in  Basic  Model 

expected  lifetime  of  component  i  =  1/A^ 
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„5  6 

Q1 =  v 


vector  of  expected  times  to  transition 
(holding  times)  given  various  states  s  and 
policy  6 

vector  of  stationary  transition  probabilities 
for  policy  5  =  Get  by  solving 


V1-^  =  £ 


=  set  of  states,  s  (2  of  them  -  some  may  be 
inaccessible  given  starting  state  and  policy) 

=  penalty  cost  incurred  when  system  fails 

=  cost  to  repair  component  i 

=  fixed  cost  for  repair  (same  no  matter  how  many 
items  (>  0)  are  repaired  at  once) 

=  vector  of  costs  given  various  states  and 
policy  6 

=  long  run  expected  cost  per  unit  time  of  policy 
6  (scalar) 

=  min  over  6  of  V_^  =  optimal  long  run  average 
expected  cost/time 

=  expected  total  cost  for  policy  6.  This  is  a 
vector  depending  on  s 

=  min  V_  .  It  turns  out  that  the  policy  which 

5  5 

minimizes  V,.  for  one  s  does  it  for  all  s 
0,s 

(see  Veinott  [ 27 ] ) - 


We  are  now  ready  to  specify  the  Markov  chain  completely  for 


the  Basic  Model: 


State  Space: 


States  =  which  components  working,  i.e.,  s  =  i^2  ’**  ^ 
if  components  i^,  i  ,  . ..,  i^  C  {l,  . n}  are  working. 


1 


2 


If  system  =  k-of-n  and  components  are  identical, 
then  states  *  number  of  components  working,  i.e., 
s  »  i  if  i  components  left  working. 

Decision  Space: 

Let  ft  =  {1,  ...,  n}.  Then  ft  -  s  *  set  of  components 

which  are  failed  in  s.  Then  the  possible  decisions 

in  state  s  are:  R_  ,  where  ft  C  ft  -  s  (ft  Is 

ft  s  —  s 

s 

some  subset  of  the  failed  components  given  state  s). 

If  ftg  *  4>,  then  *  A  (do  nothing).  If  system  is 

s 

down  in  s,  then  ft  $  (must  do  something) 
s 

Transition  Structure: 

The  transition  matrix,  :  (assume  6  :  in  s) 


(Q  ) 

Ys,sUiJs~i  a 


j€sun 


iesuft 


otherwise 


jesufts 


i 

i 
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i 

1 


Cost  Structure: 


Ki* 


0,  s 


L  +  p  +  l  K  , 

i€ft 

s 


if  system  up  in  s 


if  system  down  in  s 


where 


J 


1,  if  A  =  B 


u 


otherwise 


Objective  Function: 


*  6 
P5  R0 


-1  =  — 6  ^  forms  a  single  irreducible  ergodic  chain. 


P6 


Vq  solves  (i)  (I-qJ)  vj  =  Rj  -  qj  Vfx 


*  6  6  *  6  6  6 
(ii)  P6  qj  V*  =  P  (-rJ  +  Q*  v^). 


Q2  being  the  normalized  variance  of  the  holding  time 
distributions  in  various  states. 


A  couple  of  examples  are  given  below: 


Example  2.1  (n  =  2,  nonidentical  components,  parallel  system, 
no  fixed  charge): 
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states  (s): 

12 

1 

2 

0 

decisions  (k) : 

A 

A,R2 

A,R1 

»  ^2  * 

R12 

cost  (Rg>s): 

0 

0,k2 

0,^ 

\  +  p, 

Kl  +  K2 

k2  +  p 

+  P 

A 

transitions  (Qg)  s  Qg  12>  “  l2th  row  of  Qg 


dimensions  (k) :  0  A,R^  A.R^.R^  R^.R^R^ 

cost  (Rq  ) :  0  0,K+L  0,K+L,2K+L  K+L+p,  2K+L+p, 

3K+L+p 


where  R^  denotes  repairing  i 


units  simultaneously. 
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transitions  (Q^) : 


[• 

[• 

[• 


3  2 

1  0 

0  1 

0  0 


•] 

°] 

0 


These  two  examples  will  be  referred  to  in  succeeding  sections 
as  solution  techniques  are  developed. 


2.  No  Fixed  Charge,  V  ^  Results 

Initially,  assume  we  have  a  general  coherent  system  (identical 
or  nonidentical  components).  The  following  theorem  is  the  basis 
for  many  of  the  results  in  this  section.  It  expresses,  given 
6,  V  ^  as  a  convex  combination  of  certain  quantities  which  in  the 
case  of  no  fixed  charge,  turn  out  to  be  V  ^  for  other  possible 

y± 

policies,  say  The  minimal  V_^  must,  thus,  be  V for 

y± 

some  i,  and  since  min  V  ^  turns  out  to  be  easy  to  find,  the 
optimal  V_^  and  corresponding  V  ^  optimal  policy  can  be  obtained 
for  any  coherent  system. 


Theorem  2.1:  Suppose  we  have  the  Basic  Model  with  no  fixed 
charge,  and  a  general  coherent  system.  Fix  a  policy  6  such  that: 
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(i)  in  state  s 

8 

(ii)  The  underlying  Markov  chain  defined  by  6  is 
irreducible. 


I  a  V 

se&>  s  8 


where 


*8  1,8 

* 

p  Qi 


and 


V 

s 


I  —  +  l 

iGsUfi  Ui  iGsun 

8  8 

i:(sufi  -i)  is  i:(sufi  ~i)  is 

s  s 

"up"  for  system  "down"  for  system 


Note:  There  is  a  result  in  Denardo  [8]  which  holds  for  all 

Markov  decision  chains  which  says  V  .  is  a  convex  combination  of 
R6 

R0  s 

■  ■  for  all  s.  Theorem  2.1  expresses  V_^  as  a  convex  combination 

Ql,s 

different  quantities.  While  it  isn't  true  (or  applicable)  for  a 
general  Markov  decision  chain,  it  is  much  more  useful  in  obtaining 
results  for  the  repair  models  under  discussion. 


Note:  The  assumption  (ii)  in  Theorem  2.1  is  not  really  a 

restriction  as  far  as  finding  a  V_1  optimal  policy  is  concerned. 

If  6  defines  a  process  with  several  ergodic  chains,  C,  each  with 
C  * 

its  own  V  the  policy  6  ,  which  is  irreducible  and  has  single 
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Q 

erodic  chain  V  =  min  V  is  better  than  6.  Thus,  if  6  has 
-i  C  -l 

a  chance  at  being  V_^  optimal,  it  must  define  an  irreducible 
Markov  chain. 


Proof : 
brevity. 


Since  6  is  fixed,  we  drop  it  in  notation  for 


Thus,  it  is  sufficient  to  prove  that 


?  p.<5 


sxl,s 


V 

s 


RHS  . 


This  will  be  done  by  proving  that  the  corresponding 

* 

coefficients  of  1  _<  i  £  n  and  of  p  are  equal  for  P  Rq 

and  RHS. 


Let  SP^  *  (s  ESP\  component  i  is  repaired  in  state  s) 


SP p  “  (s  PzSP\  system  is  failed  in  s}. 


s/ESP 


p  Ro 
s  0,s 


RHS  “  £  P. 
s 


8^1,  S 


l  K*±  +  l  X  (K  4p) 
168US1  iGsun 

S  S 

i :  su  0  ' i  i:sun  -  i 

s  s 

is  up  is  failed 
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where 


1'8  l  \  * 

iesun 

s 


First,  the  coefficients  of  p: 


coefficient  p  P*R  -  I  P  -  I  I  — - 

#=&>  s€«SP  (i.t):  L  xi 

P  P  .  ,  A  J 


P-t* 


tunt'i  tunt 


til  A 

using  the  s  equation  of  P  (I-Qrt)  *=  0 

0  ^ 


jGtU  «t 

j:tunt~  je^D 

jGtUft.. 


l  l  X±\  t  P-t 

t  ietunk  ’ 


i:tunt  -  i 


=  coefficient  p  of  RHS. 
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Now,  Che  coefficients  of  K^,  and  i: 


* 

coefficient  P  R^: 


I  P 

L  •  s 

s e/j\ 


K4  coefficient  RHS:  7  A  P  Q. 

i  u  i  *s  1.3 

sriGsua  * 

s 


l 

s :  i€  s  u  n 

s 


jGsUfig 


since  RHS  without  p-terms 


I  p.a  s  l  rr 

s  iGsUfJ  yi 


l  l  ?.sXi\  s  Ki 

S  iesuflg  8 


l  I  V-sV* 

i-1  1  sriGsun  ’ 


so  coefficient  K.  of  RHS 


s :  ie  sun 


A,P  Q. 

i  ’S  l,s 


“  I 

s :  iG  8 


l 

jes  J 


where  js|  -  number  of  elements  in  s 


From  the  appropriate  columns  of  P  (I-Q^)  ■  0,  we  get 


iC^  column: 


>  -  y  — i —  /  y  p  ' 

•i  »  L,i  X..+X.  A.  .Z.  .r,  *t 


v1  1 

■  i 


th 


it,  •••  t,  column: 
1  k 


** » it,  •  •  •  t,  ■  -  ^ 


k+1 


•it,  •••t.  „  j-rj. 

1  ^  ^k+1^^’^1’* 


t  \  k+X 

*  *  V  X.+  l  X. 
i  i  tj 


l  r. 

t:tuft_ 


,urVi 


k  <  n  -  1. 


thus. 


X, 

>  +  y  i  /  y  p  \  -  y  y  P 

1  s :  j  s  I  —2  IXj(t:tUfi  *t)  s:|s|*2  t:tUfi  *’ 

iGs  8  J\  «s  c  /  ies  -s  c 


and 


X 

l  ?.±i  ...*  +  I  TT~/  l  P.t\ 

..t^i  1  k  s:  |s|-k+l  k:tUQt  J 


l  l  p.t?  Ilk  <  n-  1. 


s : | s | *k+l  t : tu  n 


iGs  *s 


Thus,  coefficient  of  RHS 
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y  y  y  p  -  p  -  y  y  p 

t=l  s :  |  s  |  =  e.  t:tu»  k*j  l  ...  tlH  i£l 

if=s  =s  IK 


l  1.  /  l  p.t  +  l  p.t 

1=1  s:|s|=i.[  t:tUft  =s  t:tUft  *s 


IGs  \  iGt 


n-1 

-  I 


l  l  ,  p-u  ...* 

k-l  i, , . . .  ,1,  /i  1  k 
1  k 


r  * 

)  P  *  coefficient  K.  in  PR.. 
L  •  s  i  0 


The  following  lemma  associates  a  feasible  policy  with  each  Vg: 


Lemma  2.2:  V  ,  as  defined  in  Theorem  2.1,  is  equal  to  VY  , 

S  1 

where  y  is  a  policy  which  has  {sUft  ~i}._  ,  as  its  chain  of 

s  ie  s  U  ft 

s 

ergodic  states,  i.e.,  y  is  the  policy:  Ri  in  suftg  ~  i  [keep  the 
set  of  components  su!2fl  working]. 


Proof:  To  get  V  ,  -  V  ,  some  s  in  Theorem  2.1,  need 
-  -i  8 

6  :  P^  “0  unless  tuft  ■  sUft  .  This  occurs  when  6  *  y. 

•  t  t  s 


Corollary  2.3:  A  V  optimal  policy  for  the  Basic  Model  with 
no  fixed  charge  will  never  involve  repair  of  more  than  one  unit  at  a 
time  in  an  ergodic  state. 


Proof :  Uses  Theorem  2.1  and  Lemma  2.2  directly.  Any  policy 
6,  which  repairs  more  than  one  unit  at  a  time  in  an  ergodic  state  has 
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5  6 

V  1  *  convex  combination  of  V  ,  s  g  ,5?.  Thus,  V°  ^  V  ,  where 

”  ■*■  8  “1  8 

v 

V  -  V',  some  y  for  which  repair  is  never  done  on  more  than 
S  “X 

one  unit  simultaneously.  □ 

The  following  theorem  which  follows  directly  from  Theorem 
2.1,  Lemma  2.2,  and  Corollary  2.3,  gives  an  exact  procedure  for 
finding  the  V  optimal  policies  for  the  Basic  Model. 

Theorem  2.4:  Given  the  Basic  Model  with  no  fixed  charge,  a 
V_i  optimal  policy  <5  is  one  for  which 

6  a 

V  ,  =  min  V 

_1  *S.9> 

system  up  in  s 


where 


A 

V 


A 

l  l 

i(=s  yi  ies 


K^p 


i;S'i^ 


i:s-iE^ 

P 


Proof :  From  Theorem  2.1,  Lemma  2.2,  and  Corollary  2.3. 


Thus,  given  any  coherent  system  with  state  space  t? 
under  the  Basic  Model,  to  find  a  V  ^  optimal  policy,  compute 

A  * 

V  and  find  the  one(s)  which  are  minimum,  say  Vg.  By 

* 

Lemma  2.2,  V  defines  a  V  optimal  policy  y  which  keeps  the 
8  “1 

set  of  s  components  working  by  repairing  any  component  of  s 
as  soon  as  it  fails. 
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Notice  that  there  will  be  many  V  ^  optimal  policies 

since  the  above  procedure  only  defines  what  happens  in  the 

ergodic  states,  i.e.,  those  for  which  the  process  once  having 

entered,  will  stay  in  forever.  If  keeping  the  components  of 

state  "s"  working  is  V  optimal,  then  all  other  states 

besides  {s  -  i}.  must  be  transient  or  unable  to  be  reached 
1GS 

from  the  initial  state  of  all  components  working  (or  would  have 
multiple  ergodic  chains,  in  which  case  the  policy  wasn't  V 
optimal).  To  determine  the  optimal  policy  for  those  states,  we 
need  results,  looked  at  in  the  next  chapter. 

The  no  fixed  charge  L  assumption  is  essential.  Although 
if  L  >  0,  a  version  of  Theorem  2.1  holds,  not  every  Vg  corresponds 
to  a  feasible  optimal  policy  in  that  case,  limiting  its  usefulness. 
Some  examples  of  specific  systems  conclude  the  section. 

Example  2 . 3  (parallel  system,  different  components  (n)) 


<*P 


V 

s 


(0) 


l 

ies 

s~i^{0} 


l 

ies 

s~i={0) 


Ki+P 


If 


| s j  >  1,  then  s~i^{0}Vies 
I  s  |  =  l  (s  =  {i},  some  i),  then  s  ~i  =  (0). 


A>  < 


Possible  min  V  (optimal  V  ,)  are: 

s  -1 


U} 


Kt  +  p 


ergodic  chain 

{0} 


policy 


^  in  0 


(keep  unit  i  working) 


s  *  ( i, j  > 


{i,j> 


\  in  3 

R.  in  i 
J 

(keep  i  &  j  working)  . 


^  K  K. 

If  s  :  [s|  >  2,  >_  —  +  some  1  &  j.  A  V_^  optimal 


3  ~  Ui  uj 


policy  is  one  which  has 


(  VP  Ki  Kj  1 

V  ,  =  min  (  min  — ' J —  ;  min  —  +  — ■*-  ; 

-1  1  •  U.  .  .  u.  U. 

I  3  3  1.3  i  3  J 


keep  j  keep  i  &  j 
working  working 


If  n  =  2,  then  we  have  Example  2.1.  Below  are  some  examples 
of  policy  behavior: 
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Suppose  n  =  3 


states 

s  :  |  s  |  >  1  1  2  3  0 

Let  6^:  A  R2  R^  A  R2 

&2:  A  r2  R!  \  (R2^ 

6  :  12  13  23  R?  R,  R, 

3  E3  R2  A 

-+■  6^  is  not  V  ^  optimal,  since  it  has  two  ergodic  chains  {12} 
and  {0}  both  accessible  from  the  initial  state. 

-*■  6 2  is  possibly  V_^  optimal,  as  even  though  it  has  {12}  and 
{0}  as  absorbing  chains,  {12}  is  the  unique  one  accessible  from 
the  starting  state. 

-*•  6^  is  also  possibly  V  ^  optimal,  and  has  the  same  V as 
62*  Vq  is  needed  to  distinguish  between  62  and  6^. 

Example  2 . 4  (k-of-n  system  (system  works  **  at  least  k  n 
units  functioning)): 

k  =  1  is  parallel 
k  =  n  is  series 
=  {sis!  <  k  -  1 } . 

Note  that  s  :  |s|  <  k  -  1  are  inaccessible  due  to  the  instantaneous 
repair  and  requirement  of  repair  in  a  failed  state. 


V 


l  7T> 


8  i£s  ui 


if  | s |  >  k 


y  Ki+P  .  if  |s|  -  k 

iesun  yi 

8 


not  defined  for  |s|  <  k  (system  down), 


Possible  optimal  V  : 


k  Ki,+" 


i  some  { i^  .  . . •  ,  ij^ }  (It  • « • ,  n  } 


has  ergodlc  chain :  { i  ,  ....  i^}  ~  i,  i  6  (i^,  ...»  i^) 
policy:  keep  (i^,  i^}  working 


or 


k+1  Ki, 


/  x  =  1  — -i  ,  some  {ix,  . 

i=l  ij 


•’  *k+l}  -  {1, 


,  •  * , 


has  ergodic  chain:  {i  ,  ....  i^+1J  ~  i,  i  £  (i^,  ...»  i^+^} 
policy:  keep  (i^,  ...»  i^+^}  working. 


True  for  1  <.  k  <_  n  -  1  (k  ^  n  (series)  by  assumptions  on  Basic  Model). 
Thus,  5  optimal  is  either 

(a)  keep  the  cheapest  set  of  k  components  working  if  p, 
the  penalty  cost  is  low  enough,  or 
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(b)  keep  the  cheapest  set  of  k  +  1  components  working 
if  p,  the  penalty  cost  is  high  enough. 


Example  2 . 5  (system  up  **  component  i  is  up) : 


=  (s j i  £  s} 

V  defined  for  s  :  i  G  s 
s 


min  Vs 
s 


K.+p 

l 


K.+p 

i  r 


L 

jGs~i 


K. 

1 


at  s  =  {i}. 


So,  a  V_x  optimal  policy  has  absorbing  state  {0}  and  keeps 
component  i  only  working. 


Example  2.6  (n  =  3  system  defined  by  the  min  cut  sets  (1), 

(13}): 


=  {23,  2,  3,  1,  0} 

V  defined  for  s  G  {123,  12,  13} 
8 


+ 


k2+p 


u 


2 
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any  of  which  could  be  min. 


So,  6  optimal  is  to  keep  {12},  {13},  or  {123}  working,  depending 
on  parameters. 


3.  Fixed  Charge  Results 

In  this,  the  final  section  of  Chapter  II,  a  brief  look  will 
be  taken  at  Basic  Model  results  for  cases  when  there  is  a  fixed 
charge  L  >  0  per  repair  decision.  A  larger  fixed  charge  is  going 
to  induce  more  multiple  repair  decisions  and  theorems  in  Sections  1-2 
involving  conditions  under  which  one  never  repairs  more  than  a  single 
unit  at  a  time  will  be  no  longer  true  in  many  cases.  However,  a 
number  of  the  other  results  from  Section  2  either  apply  directly 
or  can  be  modified  for  use  here.  The  case  of  a  k-of-n  system  with 
idendical  components  is  looked  at  in  detail  as  it  yields  a  precise 
optimal  solution  using  V  ^  only  which  can  be  compared  to  the  L  =  0 
case. 

The  basic  result  from  Section  2  upon  which  all  the  V 

results  for  the  L  *  0  case  were  based  is  Theorem  2.1,  which 

6  Yi 

expressed  V  ^  for  any  policy  6  as  a  convex  combination  of  V_^  , 

where  y^  were  policies  which  never  repaired  more  than  a  single 

component  at  once  in  ergodic  states.  An  extension  of  Theorem  2.1  to 

tne  L  >  0  case  is  direct.  Unfortunately,  the  quantities  which 

is  a  convex  combination  of,  are  not  all  V_j's  °f  certain  other 

policies,  rendering  the  result  less  useful  but  still  worthwhile  stating. 
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Theorem  2.5:  (extension  of  Theorem  2.1)  Suppose  we  have  the 

Basic  Model  with  fixed  charge  L,  and  a  general  coherent  system. 

Fix  a  policy  £  such  that: 

(i)  in  states  where  C  {it  ....  n}  ~  s 

s 

(li)  the  underlying  Markov  chain  defined  by  6  is  irreducible. 

Then 


I  « 

sE<y  s 


v 

s 


where 


a 

s 


P  Q, 

•  s  l,s 
* 

P  Qi 


and 


V  = 
s 


l 

iGsUft 

s 

i:suns~i 

is  up 


l 

i€sun 

8 

i:su&  ~i 
s 

is  down 


K1+L 


K  +p+L 

— -  if  n  #  <{> 

wi 


l 

iSs 

i:8Uft  -i. 
s 

is  up 


i:sufl  ~i 
s 

is  down 


if  *  4)  . 


Proof :  As  in  the  proof  of  Theorem  2.1,  it  suffices  to  show 

*  p 

P  R0  *  l  P.8  Qi  8  vs  (RHS)  by  proving  the  corresponding  coefficients 
of  1  <.  i  £  n,  of  p,  and  of  L  are  equal.  Those  of  and  p, 

unchanged  from  L  ■  0  case  are  still  equal  so  only  need  to  check  the 
coefficients  of  L. 
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PR-  P  R 
0  •• 


(^  .  SP  defined  on  page  30)  +  L (  \  P  \ 

P  Isg^U  J 


where  9^  -  3.  U  U  •••  U.9?  *  set  of  states  in  which  some 
1  i  n 

repair  is  performed. 


Coefficient  L  in  (RHS) 


S&3 


U 


p  Q, 

•s  l,s 


l 

iesun 

S 


=  l  p 

L,  •  S 

sG^u 


I  K 


iGsun 


l  h 

iGsijfi 


“  [  P 

L  •  s 

bg3U 


•k 

-  coefficient  L  in  P  R^  .  □ 

In  the  L  *  0  case,  one  then  has  Lemma  2.2  which  states  that  V  , 

s’ 

as  defined  in  Theorem  2.1  is  equal  to  VY^,  where  y  is  a  policy 

which  keeps  the  set  of  components  {sun  )  working.  (Whenever  one 

s 

fails,  repair  it  and,  thus,  never  repair  more  than  one  unit  at  a 
time.)  This  then  eliminates  from  being  optimal,  all  policies  which 
allow  repair  of  >  one  unit  at  a  time  in  an  ergodic  state.  If 
L  >  0,  this  is  no  longer  true;  as  the  following  example  illustrates: 


A3 


.L»PP  UP  l  |«p 


Example  2.7  [refer  to  Example  2.3]  (n  =  3  parallel  system): 


Fix  6 


If 


s  =  (1),  R2  in 
or  s  =  {0},  R^2  in 


x^i^+l)  +  x2(k2+l> 


But,  if  s  =  {12},  A  in  s,  then  V  =  X  K  +  X„K0.  Then  policy 

S  1  1  z  z 

Y 

y,  of  keeping  components  1  and  2  working,  has  V  =  X^(K^+L) 

+  X2(K2+L).  So,  if  we  pick  6  sc  that  "A"  in  state  12,  then 

y 

V^2  cannot  be  expressed  as  V ,  where  y  keeps  components  number 
1  and  number  2  working. 


Given  the  failures  of  Lemma  2.2  and  Theorem  2.1  in 
limiting  possible  V  ^  optimal  policies  in  the  fixed  charge  case, 
it  is  now  clear  that  the  finding  of  a  general  optimal  policy  would 
be  extremely  time  consuming,  if  not  impossible  without  the  aid  of  a 
computer  (see  Chapter  VI  for  suggestions).  Thus,  in  the  interests 
of  getting  a  feel  for  how  a  fixed  charge  affects  things,  we  close 
out  the  section  by  looking  at  two  special  cases  where  results  are 
obtainable.  These  are: 

(1)  identical  components,  k-of-n  system 

(2)  no  penalty  cost  (p  =  0) . 

In  case  (1),  we  can  get  a  general  optimal  policy. 

Definition:  Let  f^(i)  ”  (i+k)[l/k  +  •••  +  1/i+k-l]  -  i, 

1  i  k  <  n,  l<i<n-k  +  l,  fk<0)  4  0,  fR(n  -  k  +  1)  4  ■». 
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Notice  that 


(a)  fk(i)  +  i,  +  k 

(b)  fk(i)  -  fk(i-l)  +  i,  *  k- 

Theorem  2.6:  (optimal  policy  for  L  >  0,  k-of-n  identical 
components) . 

Suppose  we  have  the  Basic  Model  with 

(1)  a  k-of-n  system 

(2)  identical  components  (X^  H  X,  =  K) ,  and 

(3)  a  fixed  charge  L  >  0  per  repairman  visit. 

Then  the  following  are  true: 

(i)  Never  optimal  to  repair  until  k  left  working. 

(ii)  Among  policies  which  repair  when  there  are  k  left 
(label  =  repair  i  units  at  once),  R^_^  is 
optimal  **  <  L/K  <  fk+^(£)*  where 

1  <_  l  <_  n  -  k  and  fk+^(n-k)  «=  °°  . 

(iii)  Among  policies  which  repair  only  when  the  system  fails 

(k  -  1  left),  is  optimal  fk(£-l)  <  <  fk(£), 

l<H<n-k  +  l. 

(iv)  The  optimal  policy  is  the  better  of  that  in  (ii),  (iii). 

Proof : 


(i)  is  direct  from  Theorem  3.3  in  the  next  chapter.  As  we 
have  a  k-of-n  system  with  identical  components,  the 


state  and  decision  spaces  are  greatly  simplified  with 


states  =  n,  n  -  1,  ....  k,  k  -  1,  the  number  of  components 

working,  and  decisions  limited  to  the  number  of  components 

desired  to  repair  in  a  state.  By  (i),  the  optimal  decision 

is  "A"  in  states  n,  n  -  1,  . ..,  k  +  1.  In  state  k, 

possible  decisions  are:  A,  R,  ,  R.  ,  . . . ,  R  ,  ,  in  state 

l  l  n-k 

k  -  1,  they  are:  R,  ,  R~,  ....  R  ,  ,  R  .  No  state 

12  n-k  n-k+1 

i  <  k  -  1  can  be  reached  by  the  necessity  of  repair  upon 
system  failure  assumption.  The  possible  policies  then 
reduce  to  the  following  2 (n-k)  +  1  possibilities: 


policy 
i - 1 

^k-1  ^ ~  ^er£°dic  states} 
A  RL  {k-l> 

A  R2  {k-1,  k} 

A  R3  {k-1,  k,  k+1} 


A 


Rn-k+l 


{k-1, 


n-1} 


2 


«k 

6k-i 

<?=  {ergodic  states} 

R1 

never 

{k} 

1 

reach 

R2 

k-1 

{k,  k+1} 

so 

R3 

i 

{k,  k+1,  k+2 } 

• 

r 

*1 

e 

• 

v 

• 

R  U 
n-k 

a 

*1 

{k,  k+1 ,  . . . ,  n-1} 

Those  policies  in  column  1  are  those  referred  to  in  (iii) 

and  those  in  column  2  are  treated  in  (ii) .  It  remains  now 
*  s 

to  compute  V_^  for  various  policies  and  compare  them. 
Below  is  a  table  of  such  values.  Comparisons  of  column  1 
values  lead  to  result  (iii)  and  of  column  2  values  lead  to 
result  (ii);  (iv)  is  obvious.  □ 
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Figure  2.2 

t 

Table  of  V  for  6  in  Theorem  2.6  Model 


Column  1  (111) 


AR. 


AR„ 


AR 


AR 


k  +  L  +  p 
y/k 


2K  +  L  +  p 
u(l/k  +  1/k+l) 


_ 3K  +  L  -I-  p 

3  y (1/k  +  1/k+l  +  l/k+2 


n-k+1 


(n-k+l)K  +  L  +  p 
y (1/k  +  •  •  •  +  1/n) 


R1  - 


R2  - 


R,  - 


n-k 


Column  2  (11) 


-1 


K  +  L 
P/k+l 

2K  +  L 


y (1/k+l  +  l/k+2) 


3K  +  L 


u (1/K+l  +  l/k+2  +  l/k+3) 


(n-k)K  +  L 


y (1/k+l  +  •••  +  1/n) 


It  is  worth  noting  that  V alone  here  gives  us  our 
desired  "optimal"  policy  -  no  Vq  needed. 


Example  2.8  (consider  Example  2.2  from  Section  2.1): 


n 


A,  k 


2  costs: 


with  fixed  charge  L  >  0 
component  repair  K/component 


states:  4,  3,  2,  1 


mean  lifetimes 
of  components:  y  =  1/A 


policies:  AR^,  AR2»  AR^,  R^  R2  “  • 
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AR  AR  AR 

- i - 1 - £_ - 1 - - -  (l.  +  p)/K  t2CJ)  =  - 

0  f2 (1)  f 2 (2 ) 

II 

f2(0) 

R  -  R  - 

- i - 1 - - - - — - *  L/K  f  (2)  -  ’ 

0  f  3(i ) 

•  I 

f,(0) 

Compare  optimal  among  (ii)  and  (iii)  to  get  optimal. 


(ii) 

policies 


(iii) 

policies 


In  general. 


e,  L  L  l-  L  l.  L  £ 

12  3  4  5 

/“ar7^  '  ART'''  ART  s'  aC  ''XT  ARn~k+l 

- 1 — ) - ~  | - 5 - 1 - - - 1 - - - * 

0  fk(l)  fk(2)  fk(3)  fk(4) 

»l 


fkCn-k+l> 


CX. 


fk(°> 


0  Wl>  W2>  W3>  W*> 


L/R  f k+i (n-k)  «  « 


The  L/K  ratio 


fixed  charge 


is  the  key  here.  A  large 


component  repair  charge 
L/K  favors  repair  of  several  components  simultaneously  while  a  small 
ratio  favors  single  repairs.  Obviously,  a  large  p  favors  an  "R^ 
policy  while  a  small  p  favors  "AR^"  types. 
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Now,  fix  all  parameters  except  the  type  "k-of-n"  system 
to  see  how  k  and  n  affect  the  optimal  policy.  (L+p)/K  and  L/K 
are  now  fixed.  Increasing  k  lowers  f^d)  for  any  fixed  i. 

This  fact  enables  us  to  state: 

Lemma  2.7:  Given  the  Basic  Model  with  identical  components 
and  a  k-of-n  system  with  a  fixed  number  of  components,  n.  Let 
1  <  kj  <  k^  <  n.  Then  the  number  of  comDonents  repaired  in  the 
optimal  policy  for  the  k^-of-n  system  is  less  than  or  equal  to 
the  number  repaired  in  that  for  the  f^-of-n  system. 

Increasing  n,  the  number  of  components  while  leaving  k 
fixed  leaves  the  previously  existing  f^(i)  unchanged  but  does 
add  more  possible  values  of  i  and  thus  further  subdivides  the 
interval  between  the  previously  last  f^(i)  and  “•  This  leads  to 

Lemma  2.8:  Given  the  Basic  Model  with  identical  components 
and  a  k-of-n  system  with  fixed  k,  let  k  <  n^  <  n^ •  Then: 

(i)  If  the  number  of  components  repaired  in  the  optimal 
policy  for  the  k-of-n^  system  is  less  than  all 
possible,  then  the  same  policy  holds  for  k-of-n2* 

(ii)  If  the  number  repaired  in  the  optimal  policy  for  the 
k-of-n^  system  is  all  possible  (i.e.,  n^-k  in  state 
k  or  n^-k+1  in  state  k-1),  then  the  number 
repaired  optimally  for  k-of-n2  is  greater  than  or 
equal  to  that  for  n^. 
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Another  interesting  question  that  is  easily  answered  is: 

What  happens  to  the  optimal  policy  asymptotically  as  k,  n  -*•  00 
with  n  -  k  held  constant? 

Lemma  2.9:  Given  the  Basic  Model  with  identical  components 

and  a  k-of-n  system,  let  n,  k  -*•  00  with  n  -  k  held  fixed. 

Then  the  optimal  policy  is  either  AR  .  or  R  .  -  . 

n-k+1  n-k 

Proof :  Notice  that  lim  f,  (i)  =  0  for  fixed  i.  Since 

k-**>  lc 

K  51  °’  If  >  fk+l^n-lc-1^  and  >  the 

optimal  policy  will  be  either  to  allow  system  failure  or  to  get 
down  to  k  left  and  repair  all  failed  components.  □ 

Notice  that  if  L  =  0  (no  fixed  charge),  the  optimal  policy 
is  either  AR^  or  R^  -  since  in  that  case,  one  never  repairs  more 
than  one  at  a  time. 

Now  consider  the  p  =  0  case  briefly  (components  no  longer 
need  be  identical)  using  a  k-of-n  system.  Using  arguments  from 
Theorem  3.3  (next  chapter),  it  is  clear  that  repair  need  never  be 
done  until  system  failure  (k-1  left)  since  there  is  no  penalty. 
Since  =  A  if  |s|  _>  k,  only  6g,  |s|  =  k  -  1  need  be  determined, 
which  is  possible  using  only  V  -  optimality.  For  the  case  of 
identical  components,  the  optimal  policy  in  (iii),  Theorem  2.6  gives 
the  overall  optimal  policy  since  those  in  (ii)  will  never  be  optimal 
given  p  =  0. 

See  Chapter  VII  (Conclusions)  for  a  summary  of  results 
presented  in  this  and  other  model-result  chapters. 
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CHAPTER  III 


THE  BASIC  MODEL,  VQ  -  RESULTS 

1.  No  Fixed  Charge,  General  Results 

In  this  chapter  we  will  try  to  pinpoint  optimal  decisions 
in  transient  states.  The  problem  is  much  too  complicated  to  solve 
for  a  general  coherent  system  except  by  computer,  so  let's  restrict 
ourselves  to  a  k-of-n  system.  In  this  case,  it  will  be  shown 
that  one  never  repairs  when  more  than  k  units  are  working  and 
that,  with  no  fixed  charge,  one  never  repairs  up  to  more  than 
k  +  1  units  working  from  any  state  if  p  is  small  enough.  As 
V_^  determines  what  to  do  when  in  s  :  js|  =  k  -  1  and  one  does 
nothing  until  k  -  left,  the  only  unknowns  are  what  to  do  in 
transient  states  where  k  -  units  are  working.  If  k  =  1  (parallel 
system),  the  problem  can  be  solved  exactly  in  many  cases  and  a 
general  optimal  solution  obtained.  This  requires  considerable 

work  and  is  done  in  the  next  section  (2).  The  other  case  for  which 
an  exact  optimal  policy  is  obtainable  is  that  of  identical 
components.  Here,  the  state  space  of  the  model  simplifies,  causing 
every  possible  optimal  policy  to  have  a  unique  V_^,  ■*-n  which  case 
V_^  is  sufficient  in  computing  an  optimal  policy. 

To  prove  the  aforementioned  results,  we  need  the  following 

lemmas : 
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Lemma  3.1:  Suppose  we  have  a  k-of-n  system  and  the 
Basic  Model-  If  it  is  never  optimal  to  repair  more  than  one  unit 
at  a  time  in  states  s  :  |s|  =  k'  >  k,  then  it  is  optimal  to  do 
nothing  in  states  s  :  |s[  =  k'. 

Proof :  First,  from  Chapter  II,  all  states  s,  |s|  ^  k  +  1 

are  transient  in  a  V  optimal  policy  so  any  realization  of  the 
process  starting  in  s  :  |s|  =  k'  using  an  optimal  policy  must 
eventually  reach  a  state  s  :  |s|  =  k'  -  1.  This  will  happen  at 
the  first  instant  decision  "A"  is  chosen. 

The  basic  argument  is:  suppose  you  start  in  state  s  and 
follow  policy  6,  under  which  in  s,  until  a  state 

t  :  | t |  =  k*  —  1  is  hit.  Then  the  policy  y,  which  first  lets  a 
component  fail  from  s  and  then  does  everything  6  does  will  be 
better  than  6  (with  respect  to  V^) .  Since  y  never  repairs 
until  k'  -  1  units  are  left,  the  result  is  true.  To  be  precise: 
let  the  starting  state  s  =  (i^,  ...,  i^.}.  It  is  desired  to  compare 
policies  with  respect  to  Vq,  where  is  a  vector  whose  length 

is  the  total  number  of  states  and  which  satisfies: 

(O  (I  -  Q0)Vq  -  Rq  -  QjV.j 

(2)  P\V0  ’  P*(”  R1  +  °2V-1>  • 
s  * 

Let  Vq  .  =  total  expected  cost  given  start  in  s'  and  follow  policy 

a  »  g  * 

6.  It  suffices  to  compare  .  and  V„  for  any  one  s'  for  if  a 

0,6  0,y 
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3 

policy  minimizes  for  one  s',  it  does  so  for  all  s'  (see 

Veinott  [31]).  Let  s'  *  s  where  s  :  |s|  =  k'  is  our  starting  state. 

*  g 

In  comparing  ,  it  is  assumed  that  an  optimal  V  policy  and 
a  corresponding  set  of  ergodic  states  have  been  found.  We  are 
concerned  now  with  transient  states  only,  as  ergodic  states  and 
decisions  there  have  been  determined  by  V  By  equation  (1),  if 
IL  in  state  t,  t  =  (i^,  ....  i^} 


(3) 


I 

netuj 


l  A 

petuj  p 


vtUj~£ 

0 


+  K. 

1 


-1 

l  X 

petuj  p 


or,  combining 


terms 


(4) 


l 

let 


0 


+ 


l 


petui 


A 

P 


where  =  total  expected  cost  given  start  in  t,  use  S(t) 

in  t  and  follow  any  policy  thereafter. 

If  "A"  in  state  t, 


(5) 


I 

net 


Now,  pick  any  policy  <5,  which  has  Rj  in  the  initial  state 
s  and  specifies  what  to  do  each  possible  time  another  state 
s',  | s' |  -  k*  is  entered. 


\ 
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As  previously  mentioned,  there  are  a  finite  number  of  such 


possibilities.  6  can  be  represented  by  the  following  sample  path 
diagram: 


s,(s..)  =  {s  u  j  (s, )  —  i,  some  il 
21  11  l‘J(81) 


S„  = 


(sUj  'i,  some  i} 


*2ial> 


R.(S2)- 


SN(S1’ ’ * ’ ,SN-1^ 


R. 

J 


R.(si) 

31 

A 


S2(S1> 


s2(si>  ~h 


(2) 

SN(S1 . SN-1) 

A  . 

V41 

_ 

(1) 

Where: 


N  =  maximum  possible  number  transitions  before  going  to  a  state  with 
J  s  J  = k'  -  1  (i.e.,  making  the  decision  "A").  Note  N  <  °°. 


"Boxed  in"  ( 


])  states  (1),  ...»  (N)  represent  "termination" , 


i.e.,  entry  into  a  state  with  only  k'  -  1  units  working  after 
1,  2,  or  N  transitions. 

Sj(s^,  ...,  sj_^)  represents  the  state  entered  after  the  j*"*1 
transition  (if  termination  has  not  yet  occurred). 


Clearly,  this  is  a  function  of  s^,  ...»  sj_i  where 
R  R 

E  s^,  Sj  ^  E  Sj  (or  would  have  had  termination). 


-+■  j(s^)  represents  the  item  specified  by  6  to  be  repaired  in 
state  s  .  If  j(s  )  3  A,  then  termination  occurs  at  the  i**1 
transition. 

-<•  s^s^  ....  s^)  =  st-2)  U  ~  i»  1  G  sji-i 

by  definition  of  the  process  involved. 

-*•  =  {s  |  6  »  A}. 

j  j  1  s 

R.  3  \ 

-*•  s/  =  {s .  1 6  =  R,  }.  s„  =  4>  by  definition  of  N. 

j  J  sj  k  N 

=  {s .  |  6  i  A} . 

3  3  Sj 

It  will  be  shown  that  the  policy  y»  represented  by  the  sample  path 
diagram  below,  has  V®  £  V®  ,  .  Note  that  the  two  diagrams  have 
identical  probabilistic  structure  and  termination  states  (1)  •••  (N) 


s^  6  s£,  1  1  l  £  N  -  1. 


R 

S2^Sl)  =  ^S1  u^sl^  ~i»  some  i’  S1  6  al* 


SN(S1 . V^^Vl’ 


s2(si)  ~j  (s^ 


s3(s2)  =  s2(s1)  ~  i2 


R*  (s  .) 
^  3  n-1/ 


®  (S)  ~  h 

(1) 


sN(sr...,sN_i) 


By  (4) 


(6) 


0,6 


X  l  X 

y  V-  vSUJ'1  + 

■a  1*  p  0  l  \ 

pGs  H  p€s  F 


K. 

1 


-1 


pGs  1 


By  (5), 


(7) 


\  Vs-E  V-1 


'  JL  Ik'° 


pGs 


PGs 


If  N  =  1,  i.e.,  6g  =  A  V  l,  then,  using  (4)  and 

(5),  plug  Vj^  into  (6)  V 


..s~l 


(6)  V  i  and  VQ  R  into  (7)  V  i.  (6), 
(7)  become  equal  except  for  the  coefficients  of  .  Thus, 


(6)-<7)  -  V®  -  V? 

0,6  0,y 


I  A 

pgsui  f 

I  * 

pis  P 


-  1 


K. 


X.K. 

=  4-j-  >  o 

plsAp 


so,  y  is  better  than  5  if  N  *  1.  We  now  proceed  by  induction 
on  N,  the  object  being  to  prove 


(8) 


l 

w  s 


*(w)-l 

l 

J-0 


Xd(Sl)Kd(Si) 


l 


P€s , 


X 

P 
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where  w|s  is  any  sample  path  starting  from  s,  H(w)  =  length 


of  sample  path  w,  i.e.,  w  =  <s,  s.  ,  s0 . s  ,  .  >  »  d(s  )  » 

i  i  H(w;  j 

decision  at  state  of  sample  path  w.  Since  (8)  >  0 

obviously,  the  desired  result  would  then  be  proved.  (8)  has 
been  shown  true  for  N  =  1.  Suppose  true  for  N  -  1,  6  is  given. 
Let  state  t^  =  s  U  j  ^  n,  Suppose 

in  t  ,  A  G  r 
in  t^,  H  E  a,  rua  =  s 
defined  to  satisfy  the  above. 


6  :  R 


3  00 
A 


r ,  a  are 


Let  5^  be  the  policy  which  starts  in  t^,  1  G  r  with  "Rj(£)" 
and  follows  6  thereafter.  Let  y ^  be  the  policy  which  starts 
in  t^,  Her  with  "A"  and  follows  y  thereafter.  6^  has  a 
maximum  number  of  transitions  to  a  state  t  :  |t|  =k'  -  1  *  N  -  1 
so  the  hypothesis  holds  for  6^,  H  G  r. 


By  (6)  and  (5) 


l 

H€r 


t 


H 


+ 


l 

HGa 


l  > 


f-  P€tc 


P 


.Vk 
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By  (7)  and  (3) 


V“  “  K4 
0,Y  j 


yr+  I  4W*  -rV 

i  P  WErU a  l  *  Iket  l 
pes  F  pes  P\  *  PGt^  H 


l  X 
L  c 


pet 


s°,  v®  -  v® 

0,6  0,  y 


X  .K 


l 


l  x p  ner  I  xp 

pGs  p  pGs  H 


y  -Jk-  vVk  _ 

it,  J  XP  0  l  \ 
1  pet0  p  pet.  H 


X.K.  X 

-J-L+  l  -l 


l  x 

pes 


p  (ter  l  Ap  I 
pes  p  L 


v  ^  -  V  ^ 

°*\  0,Y, 


by  (5)  and  definitions 


°f  V  YH- 


X.K.  X„ 

-P-+  j  V- 

I  X1  Iter  l  kp 
pes  J  pes  F 


I  pw  I 

wit.  w  j=0 


£(w)-l  Xd(s  )Kd(s  ) 

J  ■  J - 


l  K 


pes, 


by  induction 
hypothesis 


=  l 


(ter  l  * 
pes  p 


X.K. 

tV+  i 

p4sp  wit* 


£(w)-l  Xd(s.)Kd(s.) 

Pw  l  - j~ 

j=0  pL.  p 

j 


i  pw  i 

wls  j=0 


(t(w)-l  Ad(sJKd(sJ 


I  \ 


pes. 


which  is  what  was  to  be 


proved. 


□ 


Lemma  3.2:  Suppose  we  have  the  Basic  Model  and  a  k-of-n 
system.  Let  k'  >  k.  Then,  if  it  is  optimal  to  do  nothing  in  all 
states  s  :  |s|  >  k',  it  is  never  optimal  to  repair  more  than  one 
unit  at  a  time  in  states  s  :  |s|  *  k'. 
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Proof :  The  basic  approach  will  be  to  use  the  Vq  equations 

to  show  that  if  one  starts  in  state  s,  |s|  =  k'  and  repairs  some 

subset  ft  *  {j  ,  ...»  j  ),  m  >  1  of  the  failed  components  and  then 
si  m 

follows  any  other  policy  upon  again  getting  back  to  a  state 
t  :  |t|  =  k'  (we  have  assumed  "do  nothing"  optimal  for  |t|  >  k'), 
that  this  policy  is  worse  than  at  least  one  of  the  following: 

^ft  -j  in  s,  1  <_  1  <_  m,  then  anything 
s  ° 

(ii)  A  in  s  and  R  at  the  next  transition. 

s 

This  shows  the  desired  result  since 

if  m  =  2,  any  policy  repairing  two  at  once  in  state  s  is 

dominated  either  by  a  policy  which  repairs  one  at  a  time  or  does 

nothing  in  s  =*  m  =  2  never  optimal 

(induction  argument)  suppose  ,  | ftg |  =  m  -  1  is  never  optimal 

s 

in  any  s.  But  any  ,  | ftg |  =  m  is  dominated  either  by  a  policy 

s 

which  does  nothing  in  s  or  by  a  policy  which  repairs  m  -  1  units 

in  s  **  R^  ,  | ftg |  =  m  is  never  optimal, 

s 

Now  for  the  specifics: 

Let  s  *  (i  ,  i2>  ...»  i^,);  |s|  =  k'  >  k  (k  of  n  system) 

“s  *  (J1’  J2’  ^  =  m  >  1  * 
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Now 


sUfi  m 
s 


S  g  p 

v_  R  =  vo  A  +  2  K  (since  instantaneous  repair) 

’  ft  ’  p=l  Jp 


(9)  .  I 

p=l 


.  ,  X. 

k  1 


SUO  -i  m  j  sun  ~j 


s  p 


m  0,A 


l  \  +  l  K 

q=l  q  q=l  Jq 


+  I 


pil  + VVa 
q  q 


s  -p 


Let 


til  v  - 

+  y  k,  -  v — -4 — 
p-i  JP  £‘i  +  I‘, 
q  q 


by  (5)  with  t  =  su£ 

s 


(10) 


Zs(J)  ^  1 


X  . 


%«!■  V  4  1  +  X  +  Jx  +  X  +  j> 

■  L  q  i  c 


m  j 

z  (8)41+  J  - P- 

s '  s'  —  *•_  m 

p=l 


l  x.  +  £x. 

q=l  Jq  iq 

q^p 


z  (n  ~  p) 

s  s 


Note  that  by  (5)  and  instantaneous  repair, 


Xi  sun  -i  5 


l 


P  +  l  K 


0,A>Rn  ptilx  V°>A  p.i 

8  q 


-l 


P  L  i 
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where  A  -*■  denotes  doing  nothing  in  s  but  at  the  first 

s  s 

transition  after  s. 

If  in  (9) ,  one  repeatedly  applies  (5)  to  those  terms 

left  on  the  RHS  for  which  t  D  s  until  there  are  none  such  left, 
then  brings  all  coefficients  of  v!!  over  to  the  left  hand 


side,  and  multiplies  the  whole  equation  by 


IXi  + 

_ 9-  < 

Z*i 


the  result  is:  (using  (4)  to  combine  terms  into  V  ) 

°’Rfi  -j 

s  p 


,«•>  -  l 


k  i  sU  A  '  i. 


ktf'*  8  P  V 


I‘)\, 

1=1  J,1  1  m 


p=l  p 


TT  ^  • 

V  ,  m  3 

TT  +  I  TT  T 


P  „s 


z  (a  ~  jj 


I‘1  pil  IVTT  '\i.  s  “  v 

q  9  qrp  Jq  s  P 


l 

p  q  p  q 


li 
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CO  o 


If  0.  ,  then  V*  R  >  V*  “>  not  optimal 

j.,  then  V®  >  V®  “*■  not  optimal.  □ 

°*Rn  0,Rn  -j. 

s  s  J  j 

Note :  In  the  previous  Lemma  3.2,  the  assumption 

k'  =  | s |  >  k,  the  type  of  system,  is  crucial.  If  k'  =  k,  then 

the  policy  A  -*■  (let  a  component  fail,  then  repair  same  set 

s 

of  components  at  next  transition)  cannot  be  used  without  incurring 

g 

the  penalty  cost  for  system  failure.  In  this  case, 

^s 

will  have  a  "p"  term  in  it,  causing  the  proof  of  Lemma  3.2  to 
break  down.  We  can  now  state  the  following  theorem: 

Theorem  3.3:  If  we  have  a  k-of-n  system  and  the  Basic 
Model,  then  it  is  never  optimal  to  repair  until  you  get  down  to 
k  items  left  functioning. 

Proof :  Consider  s  :  jsj  =  k' ,  k  +  1  <_  k'  <_  n.  If  k'  =  n  -  1, 
we  know  it  is  never  optimal  to  repair  >  1  unit  at  a  time  since  only 
one  unit  is  down. 

Then,  Lemma  3.1  "A"  optimal  for  s  :  |s|  =  n  -  1 

(also  for  k'  =  n  since  none  down) 

Lemma  3.2  =*  never  R  >  1  for  k'  =  n  -  2 

Lemma  3.1  **  "A"  optimal  for  s  :  |s|  =  n  -  2. 
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Continue  alternating  applications  of  Lemmas  3.1  and  3.2  until 
k'  =  k  +  1  is  reached  and  then  we  are  done,  i.e.,  have  shown  "A" 

optimal  for  s  :  |sj  =  k'  >  k  or  do  nothing  until  k  -  left.  D 

Using  the  results  of  Chapter  II  and  Theorem  3.3,  we  now  have  a 

pretty  good  idea  of  the  optimal  policy  for  a  k-of-n  system.  Since 

one  does  nothing  in  states  s  :  |sj  >  k  and  will  never  reach  any 

states  s  :  |sj  <  k  -  1,  only  states  s:  jsj  =  k  -  1,  k  need  be 

considered.  From  Example  2.5,  possible  ergodic  chains  are 

C  =  {s  ~  i;  i  G  s);  s  :  |s|  =  k  +  1  or  k.  For  states  in  C  ,  the 
s  1  1  s 

optimal  decisions  have  been  previously  described,  in  s  ~  i. 

In  other  states,  decisions  are  restricted  so  as  to  form  no  other 
ergodic  chains.  Note  that  for  non-ergodic  states  of  sizes  k  -  1,  k, 
decisions  have  not  been  restricted  to  one  unit  of  repair  at  a  time. 

It  would  be  nice  to  be  able  to  also  say  that  in  the  remaining 
transient  states,  it  is  optimal  to  never  repair  more  than  one  unit 
at  a  time.  For  k  =  1  (the  parallel  case),  this  is  almost  true  by 
the  next  Theorem  3.4. 

Theorem  3.4:  Given  the  Basic  Model  and  a  parallel  system  of 
components,  (a)  it  is  never  optimal  to  repair  more  than  one  unit  at 
a  time  in  a  state  s  :  |s|  =1,  (b)  it  is  also  never  optimal  to 

repair  more  than  two  at  a  time  in  the  failed  state  0. 
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Proof:  We  do  (b)  first. 


Let  {components  repaired  in  state  0}  =  fi  ,  |f20|  =  m. 


fjl’  j2 . jm} 

Want  to  show  m  >  3  is  never  optimal.  Take  m  _>  3. 


.  in 

(U)  V0’rq  =  Si  \  +  P  +  ^ 

“n  P“1  P 


=  P 


+  I  +  I 

p=l  Jp  p=l 


m  Xj2_  >'jP  V-1 


m  0,A  m  . 

P  P=1  IS  l\ 

q=l  3q  q=l  q 


(since  | ~  j  |  ^  2,  "A"  is  optimal  there  by  Theorem  3.3) 


(12) 


,0 


m  j 

V0  R  =  I  K  +  p  +  VQ°A  P 

’  vjP  q:1  q 

y  q^p 


n  -j  m 

V  P  =  V°  _  p  -  7  K 

“v.  a  ’• 


Substituting  (12)  into  (11)  gives 


A. 

m  J 


£_  v° 


°*Ro  p=l  ^  ’°.RVjp 


I  s 

q=l  Jq 
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U1 

I 

q  =  l  3q 


l  X.  K.  -  V  | 
q=l  Jq  -1| 


X  . 

m  J 

>  I  — ^-V 


2_  .,0 


i  m  OjR.  . 

P"1  IK  V->P 

q=l  Jq 


since 


m 


V  <_  min  X.K.  +  X,  K,  <  7  X.  K. 

””  i  lr  J  J  fClC  _  -I  J  J 

J  j  q-l.  q  q 


v°  >  v° 

V°,R  -  0,R 

0  "0  Jp 


,  at  least  one  p.  Setting  m  =  3  here 


proves  m  =  3  is  never  optimal.  Given  m  -  1  is  never  optimal, 
m  is  never  optimal  by  the  above  argument  so  (b)  is  true  by 
induction. 


Now  to  prove  (a).  Suppose  we  are  in  state  {i}  =  s. 


Let 


=  set  of  components  repaired  in  i 


The  object  is  to  show  that  an  it  >  1 

method  will  be  to  show  V*  to  be 

“i 


decision  is  never  optimal.  The 
>  a  certain  convex  combination 
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of  V-  D  q  =  1,  ....  m.  This  is  sufficient  to  prove 

u,  K. 

q 

Vn  >  „  ,  some  i  <  q  <  m  which  proves  the  desired  result 

°’V  ~  °’Rj  _  “ 

1  q 

that  repair  of  more  than  one  unit  at  a  time  in  a  state  {i}  is  never 


optimal.  It  now  remains  to  show  Vn  >_  convex  combination  of 

i  i 

V  • 


Let 


U1 

X.+  l  X. 

1  P=1 J 


^  •  P{£2  Ui  -►  j  |no  repairs} 


=  Probability  that  if  one  starts  with  components 
ftgUi  working  and  does  no  repairs,  that 
component  j  is  the  last  one  left. 


Notice  that  £  n .  =  z  (ft  )  as  defined  in  (10).  Define  n  =  £  n, 

q=l  Jq  11  -=i  J 


q=i  q 

Our  desired  convex  combination  is 


q=i  Jq 


m  nj 

<13) 

q=l  n  3, 


.  \/j„  VS  V. 

q=l  n  \  i  Jq  i  / 


using 

(4) 


ra  ni  j  ,  m  n.  /X.+X.  \ 

-  i  -4*  v*  4- 1  jsLuA . 

_ i  _  v-/  \ 


q=l  n 


n  q=l 


V? 
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1 


fi.Ui 


v0  R  =  l  Ki  +  VOiA 

0,Rn.  P=i  JP  U,A 


m  pi  /  m 

P=i  n  \p=i  py 


/  m 
V  ^  xi 

p=i  jp 


-  -=^-  [y(fi.Ui  )1 

n  A  L  J 


where  y  is 
defined  below 


Some  notation  is  now  needed: 


Let 


X.  X.  X 

y(ijk)  4  1  +  ,  1,  -  +  T'ir-  + 


X  . +A,  X  ,+A,  X  ,+x . 
J  k  1  k  1  j 


X. 

m  Jn 

y(V  41+  I  T1 
p‘l  l  x 


q=l 

q^p 


be  a  recursive  definition  for  y. 


Let 


bi(Ri’  V  =  U - 

l  h 

k=l  k 


setting  jm+1  =  i  , 
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bi<yq’  v 


m 

n  \ 

r=l  Jr 
r^P.q 

A .  +A . 
jP  3<i 


m+1 

y  — - — 

r3  1  Jp  J,  j 

j'p.q 


m+1 

I 

r,  =1 


r3  ^  A .  +A .  +  £  A . 
^P.q.r,  Jp  Jq  t=3  Jr 


m+1 

I 

rc=l 


5  A  +A.  +  l  A. 
^P>q.r3,  Jp  ^ q  t=3  Jrt 


m+1 

‘  l 

r  =1 
m 

^p»q»r3,. 

rm-l 


in 

a .  +x .  +  y  A . 

• »  ^  p  ^  q  t— 3  ^  r 


If  Ai  -  ni*  order  jj. . 3m  so  that  {jx*  •••*  =  Aitk  £ 


Define 


Note 


y(nt  U  i) 


=  zi(^i)  +  \±  l  bi(s,  i^) 


sg^ 

s  I  >2 


n  +  Ai  l  b  (s.  Q  ) 

I  s|  >2 


so. 


(14)  V 


O.R, 


Q,  p 


.  A .4-  I  A. 

I  -?V-P+[  [  K  )  — F-1-3j- 

=1  n  \p=1  jp/  n-A 


5  J-  Jp  /  5 


V  V 

-1  V-1 


i  n  sO^ 
1  s  I  >2 


l  b  (S,  n  ) 

■**  A 


want  to  show  (14)- (13)  >  0. 


(14)-(13)  = 


V 


l,  Kj  (h  *  \  xj  -  "j  (\  * 

p=l  Jp\  q=l  Jq  Jp\  JI 


-V_1*Xi  l  b1(s,  Q±) 


sG3^ 
I  s  I  >2 


A  lot  of  algebraic  manipulation  leads  to: 


(15) 


Ki  ( *i +  l,  *j  -  "j  (h +  Aj 

p\  q=i  q  p'  i 
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Thus  (14)-(13) 
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>  0  since 


l  ‘kK*iV-> 

lsli2 


□ 


It  would  be  false  to  say  that  one  never  repairs  more  than 
one  at  a  time  in  state  zero. 


Example: 


n  =  3 


parallel  model 


^  -  3  x2  =  2  *3  =  -5 


Kx  =  1  K2  =  2  K3  =  20 


P  =  2 


V  _^  =  min 


min  X  (K  +  p), 
i=l,2,3 


min  X . K .  +  X . K . 

<i,j>  11  J  J 

=12,13,23 


min{9,  8,  11,  7,  13,  14}  =  7 


So , 


&  =  { 1 ,  2 }  and 


«?pt  -  R„ 


.opt 


Theorem  3.3  =»  6°pt  =  6°pt  =  6°pt  =  A 

Theorem  3.4  =*■  5°pt  G{ R  ,  R„,  A} 

+  J  i  z 

need  single 

ergodic  chain  6°pt  e{R12,  R13,  R23>  . 
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From  state  3,  you  can 


(i)  R, 


(ii)  R2 

(iii)  A  -*■  R12,  Ri;j  or  R23 


vn  D  =  "  5 

0 1 R^ 


=  -  3 


V  5 

O.A-R^  =  -  7^ 


(i),  (ii)  are  not  optimal,  giving  an  example  of  a  problem  for  which 
it  is  optimal  to  repair  two  items  in  state  zero.  Notice  the  negative 
Vq's  .  Although  a  optimal  policy  minimizes  the  total  expected 

cost,  the  Vq's  themselves  are  not  the  total  expected  cost  given 
a  starting  state  for  that  would  have  to  be  ^  0  since  there  are  only 
costs  in  this  model,  no  benefits. 

The  remaining  question  is:  Does  Theorem  3.4  hold  for  k-of-n 
systems  when  k  >  1?  In  general,  the  problem  is  still  open.  There 


is  no  "nice"  convex  combination  of  V, 


ili2'”ik 


's  which  is 


smaller  than  V 


i  i.  •  •  •  i 
X1  2  \ 


given  m  >  2  for  k  >  1  as  in 


Theorem  3.4.  We  can  say,  that  if  p  is  small  enough,  part  (a)  of 
Theorem  3.4  holds  for  k  >  1  using  the  method  of  proof  of  Lemma  3.2. 


Lemma  3.5:  Suppose  we  have  the  Basic  Model  and  a  k-of-n 
system.  Then,  if  p,  the  penalty  cost,  is  less  than 


k 

I A  Ki 


where 


A .  K  <  A  K.  ,  „ 

‘i  h  h  h  ‘  Vi  • 

n  n 
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it  is  never  optimal  to  repair  more  than  one  unit  at  a  time  in  a 
state  s  :  | s |  =  k. 


Proof :  Pick  s  =  (i^. 


ik)’ 


n  =  (j 


V  J2* 


m  >  1 

The  proof  of  Lemma  3.2  holds  exactly  up  to  Equation  (10')  where  a 

p"  term  must  be  added  to  the  RHS  due  to  system  failure  when 

following  the  policy  A  •+•  fl  .  The  RHS  is  now  only 

m 


>  0 


<  E 


=  1  JP  J 


k’ 

l  \ 

q=l  q 


which  is  true  given  the  assumption  on  p.  □ 

In  the  next  section,  we  look  at  the  parallel  case  in  detail; 
coming  up  with  an  exact  optimal  policy  in  some  cases,  by  defining 
optimal  decisions  in  states  0,  1,  ...,  n  which  are  transient. 

One  final  note  with  respect  to  fixed  charges.  Recall 
Theorem  3.3  which  states  that  for  a  k-of-n  system,  it  is  never 
optimal  to  do  anything  until  there  are  k  or  less  components 
working.  The  proof  of  this  theorem  (and  thus,  those  of  Lemmas 
3.1  and  3.2)  does  not  require  L  =  0  and,  in  fact,  goes  through 
with  only  slight  modifications  to  include  an  ”L”.  Theorem  3.3 
is  restated  below  now  including  specific  reference  to  the  fixed 
charge  case: 
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Theorem  3.3  (restated):  If  we  have  a  k-of-n  system  and 
the  Basic  Model  with  some  fixed  charge  L  0  for  repair,  then  it 
is  never  optimal  to  repair  until  you  get  down  to  k  items  left 
functioning. 

Proof :  Same  as  for  Lemmas  3.1,  3.2,  Theorem  3.3,  only  with 
an  "L"  added. 


2.  The  Parallel  Case 

In  Chapter  II,  a  V_^  optimal  policy  was  found  for  the 
Basic  Model  and  any  general  coherent  system.  Unfortunately,  a 
V_^  optimal  policy  is  only  unique  up  to  what  goes  on  in  the  ergodic 
states  which,  at  least  for  the  Basic  Model,  comprise  very  few  of 
the  total  possible  number  of  states.  Thus,  to  break  ties  among 
V optimal  policies,  and  find  the  optimal  decision  in  transient 
states,  Vq  was  looked  at  in  the  previous  section  for  k-of-n 
systems.  Using  it,  the  possible  decision  space  was  restricted 
significantly,  but  the  exact  optimal  policy  given  any  possible 
values  for  p,  K^'s,  and  A^'s  was  to°  comPH-catec*  computationally 
to  be  found.  Given  specific  parameter  values,  optimal  policies 
can  be  found  using  linear  programming  or  policy  improvement 
algorithms  on  the  computer  as  in  Chapter  VI. 

There  is  one  case,  that  of  the  parallel  system  (k  =  1), 
where  a  general  optimal  policy  form  can  be  found  for  certain 
parameter  values.  Such  results  being  the  most  desirable,  this 
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whole  section  will  be  devoted  to  the  further  restrictions  obtainable 


on  optimal  policies  in  the  parallel  case  using  the  Basic  Model. 

General  optimal  policies  will  be  obtained  where  possible. 

Consider  a  parallel  system  with  n  independent  components: 

1,  2,  ...,  n,  with  exponential  parameters  X^  >  >  •••  >  X^,  i.e., 

order  them  by  increasing  mean  lifetime.  Let  i^,  ,  ....  i^  and 

j i ,  j^,  •*.,  represent  orderings  of  the  components  by  increasing 

repair  costs  K.  <  K.  <  •••  <  K.  and  by  X.  K.  <  X.  K.  <••♦<  X.  K. 

i.  i-  i  J.  In  Jo  Jo  J  J 

Li.  n  it  4  z  n 

for  repair  cost/mean  lifetime  ratios  respectively. 

By  the  results  of  Chapter  II,  we  know  that 


V  ^  =  min 


min  X.(K.  +  p) ,  min  (X.K.  +  X.K.) 

1-1 . n  11  (i,j)e  11  33 

1 , . . .  ,n 

i+j 


=  min 


min  X1(Ki  +  p) ; 
i-1 , . . . ,n 


X.  K.  +  X.  K. 
J1  J1  32  J2 


There  are  two  types  of  optimal  policy  structures: 

(I)  V_x  =  X^(K^  +  p),  some  i.  Then  <?  =  {ergodic  states} 

=  {0}  <$°pt  =  One  then  has  to  determine  C  for  i  =  l,...,n. 

We  know  that  6°pt  =  R  ^  (where  R^  =  A)  since  it  is  never  optimal 
to  repair  more  than  one  unit  at  a  time  in  i  (Theorem  3.4).  Thus,  if 
start  in  i  and  R  the  next  new  state  entered  will  be  p(i). 
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Define  a  policy  6(i)  by: 


(1) 


6(1>  =  Rp(i) 

6(p(i))  -  R  , 

p  (i) 


6(P^(i»  -  R  1+1 

P  (i) 


lc+1  k  n  f 

where  p  (i)  =  0(6(p  (i)  =  A),  some  k  <  n  and  p4(i)  f  p  (i) 


if  r  f  q  since 
chain. 


{0}  must  be  the  single  irreducible  ergodic 


(II)  V  ,  -  \ .  K.  +  A.  K,  .  Then 
1  3i  jl  j2  32 


&  =  (jj..  j2> 


6°pt  =  R. 


6opt  =  R. 


Must  determine  6°pt,  i  i  j  ,  and  6°P 


<5°pt  =  R 
6°Pt  -  R 


...  as  before  where  =  A  and  0  <  p(i)  <  n. 
P (i)  0  - 


0 


P1P2 


since  repair  of  a  single  unit  would  produce  two 


erodic  chains  and  repair  of  three  or  more  units  is  impossible  by 
Theorem  3.4  (b). 

First,  let's  examine  policies  of  type  (I).  Suppose 


V  .  =  A  (K  +  p).  A  policy  6  can  be  characterized  by  partitioning 


the  set  {1,  ....  n}  of  components  into  m  subsets  where 

m  =  number  of  i  such  that  in  any  state  j.  Call  these 

components  a,,  a„  ,  a  ,  ordered  by  a u~  •'•••■  .j  .  Let 

i  i  m  1  Z  m 

c.  =  {1  j  £  n|pk(a.)  =  j,  some  k}.  Within  each  set  c  ,  we 

2  V 

can  find  an  ordering  a^,  p(ai>,  p  (a^,  .  ..,  p'(a.)  some  k, 
k+1 

p  (a.)  =  0.  Order  the  elements  of  c.  in  such  a  fashion  v.. 

i  l  l 

c.U  •••  U  c  =  {1,  n)  but  c .  n  c .  f  d>  necessarily  if  i  f  i. 

1  m  i  '  1  j  J  J 

Now,  let  6  =  c^c 2  •••  c^  and  suppose  we  are  in  state  i  (the 
first  state  hit  for  which  one  component  is  left  after  doing  nothing 
since  all  components  were  up).  Let  p  be  defined  as  in  (1). 


Using  (2)  for  s  g  &  with  the  single  equation  (3)  gives  the 

g 

solution  (unique)  for  V^,  s  €=  &  .  This  is  the  same  for  all  possible 
<$'s  since  the  6's  being  considered  are  already  V  .  optimal 
Thus,  equations  (2)  for  s  ^  (transient  states)  are  sufficient  to 
differentiate  between  <S  and  some  other  V  ^  optimal  policy.  By 
Veinott  [31],  minimizing  for  one  s  does  it  for  all  s.  Choose 

s  =  { i } .  As  the  s  €E  {j}^  equation  of  (2)  will  never  depend  on 
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t  :  1 1 1  >1,  only  the  rows  1,  2 .  n  of  I  -  are  needed 

for  computing  V^,  any  i. 


I-Q0: 


row  decision 


s  >1  i 


Columns 


0 


R. 

3 


X.  -X. 

l  l 


x.+x.  x.+x. 

13  13 


1/X, 


1/(X±+X  ) 


Equation  (3)  ^  +  p. 


Let 


Z6  = 


T-  ~  Qr 


rows  s  =  1 ,  2 ,  . . . ,  n 

columns  s  =  1,  2,  ....  n 


is  invertible  (elements  given  previously), 


Equation  (2)  =>■  for  1=1,  n, 


(A) 


V5  =  Z  ^ (R^  -  Q5  V6  ) 

o  s  ^  o  yi  -r 


r  -  -i  -th  c  -1  .U 

Let  z.  .  =i  row  of  z.  then 
0,1*  o 


(Ai) 


vo,s  ~~  ~  <,i  ^ 
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If  6 (i)  =  A,  then 


<  o* 


•o 


i 

1  o* 


or 


V 


i 

0 


If  i  ,  some  j  with 


6(i)  =  RP(i) 

<5  (p (i) )  =  R  , 

P  (i) 


'5(pk~1(i))  =  R. 


6(p  (i)  =  j)  =  A  as  in  (1)  , 


then  z 


-1 

6,i< 


P(i) 


P2  (i) 


A.+A  ...  Xp(i)  X  2,.,  X  2,.,  X  3... 

=  (Q...0  ~  -El1! _ 2_iil  jJ.1)  .  P  JO. 

'  A.  A  ...  A  „ 

1  P(l)  P2(i) 


•  Pk-1(i)  Pk(i)=j 

X  k-1  +X1 


k-1... 
P  (i) 


(ro  -  q5  v$)C 


p(i) 


V-1 


V(il '  vwVw'  Vi)+v<i) 


K,  - 


P^CQ 

V-1 


Pk(i)=3 

"V-l 


P  i(i)  J  J 
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so. 


<5>  V0,J-  l 


n=l 


/X  .  +X  \  „ 

n-1...  n. .  V  V  , 

* — ^VP  (i)-h  K 

pn-i(»  pn(i)  / p  a>  p  <o  Pn(i) 


-i 


or 


.  tC 

v1  =  y 

0  (S  <- 

’  n-1 


P  (i) 


+  V<*>) 


-Oil.  V 


Pn(i) 


Pn(i) 


-  V- 


where  p  =  t~  =  mean  lifetime  of  component  i. 
i  Ax 


We  can  now  compare  various  policies,  starting  from  state  i: 


Definition:  Let  i  -*•  p(i)  •+•  p2(i)  pk(i)  =  j  -*•  0 


l-l 

denote  the  policy  which  has  <5 (p  (i))  =  R 


P*(i) 


Lemma  3.6:  For  the  Basic  Model,  parallel  case  with  no  fixed 


charge,  if  >  A^  and  for  some  i,  j,  then  unit  i 

will  never  be  repaired  in  an  optimal  policy. 


Proof :  First,  we  will  show  that  one  never  repairs  number  i 
in  the  ergodic  states  and  then  in  the  transient  states.  To  use  in 
an  ergodic  state,  we  need 
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£  =  {0},  R.  in  0 
1 


(V-1  =  Ai(Ki  +  p)) 


or 


&=  {ik},  in  k 

\  in  1 


(V-1  -  Vi  +  w> 


Note  X.  >  X.,  K.  >  K.  ^X.K.  >  X.K..  But  X  K .  >  X.K. 

i  J  1  J  ii  11  ii  ]  ] 

X .  >  X  (so  X^p  >  Xj  •  p  for  p  >  0) 


=*■  X.(K.  +  p)  >  X.(K.  +  p)  so  £=  (0}  .  _  loses 

ii  i  i  K .  in  u 

j  j  l 

to  S  -  (0}_  .  „  and  is  never  V  ,  optimal. 

R.in  0  -1 

1 

Similarly,  since  X >  X ^ ,  XJC^  +  X^K^  >  X^K^ 
=*•  &  =  {jk}  beats  &  =  {ik} 

=>  never  repair  component  i  in  an  ergodic  state. 

For  the  transient  states  {£},  let 


6  <  y 


6  :  Rj, 

in 

then 

Y  :  Rj 

in 

then 

t: 

i ,  we  get 

if 

al  = 

°  (! 

- 

K, 

-i 

(w<l  +  lJi) 

i 

Pi 

-  V-1 

‘  pi 

ll  “2 


then 


(v, 


+  \t  )  -  V  • 

V  ^  -1 


and 


+  Vk 


-*■  0  . 
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h  Kj 

which  never  occurs  since  —  >  and  p^  <  P^  and 
so,  l  -*■  i  -*■  0  never  optimal. 


Now 


6  .  „6 


suppose  a^  5*  0,  a2  =  0.  Again  using  (5),  6  <  y  **■  VQ  <  VQ 


-  “ 

K 

^i 

+  v±)  ~  V-1  *  Wi 

+ 

(„i+  ,  A.v_i  . 

1  al  1 

_  _ 

_  — 

K. 

K 

a. 

+  V  ^  ‘  '  "J 

_  _ 

+ 

<“i  +  w  j  —  -  v_,  •  p 

1  ax  1 

(6) 


^%b^Vv"4-i'\)<0 


J 

>0  >0  >0 
K 


Notice 


that,  if  V  i  _>  “ — >  then  this  is  impossible  and  6  is 


never  optimal. 


6  ..<7 

Consider  policy  a:  i  0  (A  in  i)  <5  <  o  **  Vq  < 


Ki 

(U£  +  V  ^  "  V-l»*i 


(yi  +  v>r-  v-i  *  v 

1  aL  1 


<  0 


~  (u*  +  vi>  7[+  <vi  + 


p_  "  V-1 


<  0 
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K 

ai 

<f>,  if  V  <  -  and  6  is  never  optimal. 

^a^  (H-^i-M^O) 

Now,  suppose  we  have  6  =  l  -*■  i  -»•  -*■  •  •  •  a^  -*■  0,  k  _>  2 ,  a  as 

before,  and  y  =  il-*-j-*-a^->***-^a^-*-0,  6  <  y  **  (6)  holds  so, 

K 

al 

as  with  k  =  1,  V  ,  >  — —  =*■  6  >  y. 

-1  —  p 

al 

6  <  a  «  Vq  <  Vq,  (using  (5)), 


K 


(W*  +  Ui)  IT  -  V-1  *  yi  +  (yi  +  ^  •  V-1  *  Pa 

i  1  a^  l 


k 

+  l 

j=2 


K 


(p  +  p  )  — ■*-  -  V  •  p 

Vi  aj  V  _1  “j 


<  0 


K  k 

+  V  v  +  l  *a 

i  j=2  j-1 


K 

a 


+  <-*  +  v 

j 


+ 


K 

ai 

<P,  if  v  x  i  •^—L  ;  1  £  j  £  k  . 
aj 


But,  from  Lemma  3.7  which  immediately  follows,  for  5  to  be  a 
possible  optimal  solution,  we  need 
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Thus, 


K 

al 

**  V  1  <  — A,  1  <  j  <  k  -*  6  > 

"“j 


o  . 


Conclusion: 


>  y 


6  >  cr 


so,  6  is  never  optimal.  So,  never  repair  i  in  any  state.  □ 

Henceforth,  in  addition  to  X^  >  •••  >  X^,  I  will  also  assume 
<  • • •  <  Kn»  which  avoids  inclusion  of  "irrelevant"  components, 
i.e.,  those  which  are  known  to  never  be  worth  repairing  from  the 
start. 

The  next  two  lemmas  are  useful  in  further  reducing  the 
possible  optimal  policies: 

Lemma  3.7:  Suppose  we  have  the  Basic  Model,  parallel  case 

with  no  fixed  charge  and  X^  >  X£  >  •••  >  X  .  Suppose  we  start  in 

state  i.  Then  i  -*■  a  beats  i  -*  b  -►  a  (i  -►  a  means  R  in  i) 

a 
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i  ^  Ufe  i  \  vA 

K,  K,  >  0,  if  >0 
b  >  i 


U,  b, 
31  J2 
if  b*j. 


<  V 


-1 


4> ,  if  b  >  i  or  b  f  j.  so,  b  >  i  =»  6  <  y 

&  b  j  jx 


□ 


Note:  Lemma  3.7  works  for 

6  :  c.  ->•••->  c  -►  i  -►  a  -*■  a,  -+•••-*■  a  0 
1  q  1  p 

Y  •’  c,  c  •*i-*-b-»-a-*-a,  -*••••-*-  a  0 

i  q  i  p 

Y»  5  ' 

since  y  <  &  **  <  VQ  where  y'  :  i  -»■  b  a,  6'  :  i  -►  a  . 

Lemma  3.8:  Suppose  we  have  the  Basic  Model,  parallel  case 
with  no  fixed  charge.  Suppose  we  start  in  state  i.  Components 
are  ordered  yi  <  y2  <  ”*  <  yn  ' 

(Xi  >  X2  ”  *•*  "  V 

Then: 

(i)  i  -*■  0  beats  i  -*■  p(i)  ->-0  if  p(i)  <  i  and  p(i)  / 

(Recall:  j1  =  argmin  A  K  ) 

j  *  i 

and 

(ii)  i  •+  0  beats  i  p(i)  -*•••-*-  pk(i)  -*■  0,  k  >_  2  if 
pk(i)  <  i. 
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Proof : 


(i) 

Let 

6 

:  i  -  0, 

y  :  i  •+■  a  0,  Vq  <  vj 

u .  \ 

K 

~  v_!  <(1  + 

u  /  u 

,  always 

true  if  a  <  i  and  a  ^ 

\ 

a/ 

a 

(ii) 

Let 

6 

:  i  0, 

y  :  i  -*•  p (i)  pk(i)  -*■  0,  K  ^  2 

k 

0<  I 

j=l 


(V_1(i)  V(i))U 


K  . 

PJ(D  _ 


PJ<i)> Vd)  U")<i>v'1 


V-1  l  v  ■  <  l  (v  1  +  w  J  \ 

j=l  pJ(i)  j=»l\  pJ  (i)  P  (i)/ 


Pj(i) 

1  i 

PJ(i) 


,  where  p  (i)  =  i 


/k-1  \  k-1  / K  j 

V-l(  ^  w  j  +  u  k  )  <  I  w  j  ( 

\j=l  pJ(i)  p  (i )/  j=l  pJ(i)\  u  j 


K\ji  +  K-j£h±i\+v  hsu 

Pj(i)  V+1(i)/  iUp(i) 


+„  .  v»> 

p  <»  Vd) 


<*■  v 


i  k,  .  \  k-1  ** 

/l  +  _£_ii) - \  <  y  ..£.J  (i). 

-if1  +  k-i  1  £  k-i 

31  i» 


i « j 
J*i  pj 


CD/  j=l  PJ(i)  L 


K  1  K  1+1 

PJ(i)  ,  P3+1(l) 

V(i)  V+1(i) 


I 
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P 


k-1 

l  »  i 

J-l  PJ<i> 


Jill 

Up(i) 


Pk(D 


k-1 

l  w  j 

J-l  P3(i> 


K  k 

.  sl lill 

p  k 

P  (i) 


True,  if 


W 


pk(i) 


-1 


K  ^  k. 

Pd)  ,  . 

yp(i) 


K  k 
P  (1) 

W  k 

P  (i) 


since 


V  ^  convex  combination  of 


K  K 

—  +  — J- 

Ui  ^ 


So,  true  if 


V  ,  <  - 
-1  P 


pk(i) 


Mil 

JP(i) 


K  k 
P  (i) 

W  k 
P  (i) 


which  is  so  as  i  >  p  (i)  and 


v  <  Jiil  + %1  . 

yp(i)  V(i) 


□ 


Using  the  results  of  the  previous  two  lemmas,  we  can  now 
state  the  basic  theorem  describing  optimal  policies  in  the  parallel 
case.  Although  it  does  not  specify  the  optimal  policy  in  general, 
it  brings  the  number  of  possibilities  down  to  only  a  few,  which 
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could  be  differentiated  by  hand  using  (5).  In  some  special 
cases,  more  can  be  said  and  optimal  policies  described  exactly. 
Such  is  the  case  when  (i)  the  cost/lifetime  ratios  are  ordered 


the  same  as  the  mean  lifetimes,  I  —  < 

w 

vp 

value  in  V  =  -  is  small. 

-1  n. 


or  (ii)  the 


Theorem  3.9:  Given  the  Basic  Model,  a  parallel  system 


and  no  fixed  charge,  let  6  =  c^c^ 


c^  be  an  optimal  policy 


as  described  on  page  77.  The  elements  of  c^  =  < i,  p(i), 

2  k 

p  (i),  ...,  p  (i)  =  a^  )  have  the  following  three  restrictions: 


(1)  pJ(i)  <  pk(i)  V  0  <  j  1  k  -  1 


(2)  i  >  p (i)  >  •••  >  pk  1(i) 


K  k  2  ^  k 

(3)  iSii  < -E-iil  <  ...  < 


P  (i) 


P  (i) 


p  k 
P  (i) 


Proof : 

Directly  from  Lemmas  3.7  and  3.8. 

□ 

Note: 

Conditions 

(1)  and  (2)  state  that. 

given  state 

i 

is  the 

first 

one  reached 

with  |s|  =  1,  the  lart 

element 

repaired 

before 

doing  nothing  must 

have  longer  expected  life  than 

i 

and 

all  the  ones  in  between  must  have  successively  shorter  ones  up 
to  the  last  one. 


90 


Theorem  3.9  simplifies  considerably  in  the  case  that  the 
component  cost/mean  lifetime  ratios  are  ordered  in  the  same  way 


/  K1  K2  Kn\ 

as  the  mean  lifetimes  (i.e.,  —  <  —  <  •••  <  — 1  condition  (3) 

\  M1 

2  k 

becomes  p(i)  <  p  (i)  <  '••  <  p  (i). 

Corollary  3.10:  Given  the  hypotheses  of  Theorem  3.9  plus 
K  K,  Kn 

—  <  —  <  •••  <  —  a  -et  c.  will  have  no  more  than  3  elements, 

U1  y2  J 

i.e.,  k  £  2 . 

Proof:  Suppose  k  >  2.  Conditions  (2)  and  (3)  in  Theorem  3.9 
lead  immediately  to  a  contradiction.  □ 

Thus,  in  this  case,  given  initial  |s|  =1  state  i,  three 
things  can  happen  before  doing  nothing  and  entering  the  ergodic  state 
zero : 

(i)  i  -*•  0  (do  nothing) 

(ii)  i  -*■  p(i)  -*■  0,  p(i)  >  i  (repair  a  component  >  i,  then 
do  nothing) 

(iii)  i  -*■  p(i)  -*■  p2(i)  ■*  0,  p(i)  <  i  <  p2(i)  . 

2 

Which  of  these  three  and  what  exact  p(i),  p  (i)  give 
optimality  can  be  easily  determined  by  comparing  ^'s  using 
equation  (5). 


Another  type  of  optimal  policy  restriction  occurs  when  the 
K„+P 

value  "8."  in  V  =  -  is  small. 

-1 

Lemma  3.11:  Suppose  we  have  the  hypotheses  of  Theorem  2.13 
K  +P 

with  V  ,  =  -  .  Then 

-1 

(i)  If  i  8  ^  j ^ ,  i  ->  0  beats  i  ■*  j  -*•  0 

(ii)  If  l  =  1,  i  ->  0  beats  i  -*■  j  -»■  0  unless  j  =  1 

in  which  case  i  1  -*  0  might  beat  i  0. 


Proof : 


i  -*■  j  ■+  0  beats  i  -*•  0 


Suppose  this  is  true.  Then 


K.+p  /  u .  \  K . 

— ^ —  >  v  >[l  +-i  1-1 

"j  -1  l  “J  /  “j 


by  definition  of  V  ^  . 


K.  K. 

D  >  U  •  — >  U  •  — 

H  Mi  u,  -  vj. 

j  J 


K. 


if  i  >  Z  or  -2-  >  — 


P8  WJ 


But,  also 
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Thus , 


So,  i  >_  4  4  j  =»  CONTRADICTION  Q.E.D.  (i). 

If  4  =  1,  then  i  >_  4  automatically. 

If  j1  4  1,  then  CONTRADICTION,  as  before. 

If  j  =  1  =  4  and  j  4  1,  get  CONTRADICTION.  Q.E.D.  (ii).  □ 

Using  Lemmas  3.7,  3.8  and  3.11,  some  special  "small  4"  cases  have 
particularly  "nice"  optimal  policy  forms. 

K  +p 

Case  I:  v  ,  =  - —  (4  =  1) 

-  -1 

A.  4  1.  Then  Lemma  3.11  =*  i  -*■  0  beats  i  -♦  j  -*•  0  V  i,  j. 
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i  -+  p(i)  -*•  *••  -*■  p  (t)  -*•  0 


i.c.,  d^_^t^ngMln^llastates=_l=^s_ogt^Mil. 


B.  j  *  1.  Lemma  3.11  **  i  -*■  0  beats  1  j  ♦  0  if  j  #  1. 


So, 


1  p(i)  p  (i)  -*■  0 


>  1  -*■  p(i)  -►  •••  -►  pq(l)  1  -*■  0 


Lemma  3.7  i  -*■  1  beats  i  p(i)  •+■  1,  since 


Ki  K  .. . 

1  <  -Elil  (, 


UP(D 


1), 


so 


(rrr3or(0). 


Thus,  given  state  i,  the  optimal  action  is  to  either  do 
nothing  or  to  repair  component  number  1.  If  i  -  1,  then  do 
nothing  is  optimal. 


i-*-0<i-*l  +  0 
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If 


/  Wi \  K1  /  Wi+1  \  K1 

then pollcy 


I  2  3  •••  i  i  +  1 


A  R^  R^***  R^  A 


is  optimal  where  u 


n+1 


Case  II:  V 


k2+p 


-1 


a  -  2) 


(<) 


A.  j4  2.  Lemma  3.11  "*  i  0  beats  i  +  j  ♦  0  y  i  >  2,  j 


so. 


i  -*•  p(i)  pk(i)  -*•  0 

>  i  -*•  p(i)  -*■•••-►  pq(i)  -*•  1  0  (if  i  i1  1) 

>  i  -*■  0  by  Theorem  2.13  (if  i  y*  1) . 

If  I  ■  1,  then  true  just  by  Lemma  3.11. 

So,  dowtMnginallstatesilsoptl^. 


B.  -  2.  i  ■+  0  beats  i  -*•  j  -*■  0  if  j  +  -  2,  i  >  2. 
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rr 


So, 

i  -*•  p(i)  -*■•••-*■  pk(i)  -*•  0 

>  i  -*•  p(i)  -»■••••*■  1  0  or  i  -*•  p(i)  ->•••-*•  2  -►  0 

>  i  •+•  0  by  Theorem  3.13  >  i  +  2  +  0  by  Theorem  3.9 

since  »  2 

>  i  -*■  0  if  i  >  2  . 


So,  the  optimal  policy  is: 

If  in  i  _>  2,  do  nothing 

If  in  i  =  1,  do  nothing  or  repair  item  two,  depending  on  how 

large  V_^  is. 

As  l  gets  larger.  Lemma  3.11  eliminates  fewer  policies.  Improving 

little  from  Theorem  3.9.  Even  in  the  case  Jt  ■  3,  it  is  no  longer 

true  that  i  l  means  do  nothing  in  all  states  1  is  optimal. 
Also,  notice  that  Lemma  3.11  and  its  applications  do  not  require 

K1 

any  particular  ordering  on  the  — ^'s,  only  knowledge  of  the  minimum 

Wj 

K±+P 

— - —  .  This  concludes  investigation  into  V_  optimal  policies  when 

U 

K  +p 

V  -  — —  ,  some  i. 
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Now  suppose 


\  \ 

V-1"  V 
•'1  ■'2 


where 


Ki  K1 

j.  -  argmin  -*>■  ;  j,  -  argmin  -**■ 

1  J  wi  2  WJ 


Suppose  you  stare  In  state  i.  A  policy  Is  then  of  the  form 


(1)  1  p(i)  ■*•••■*• 


P  (i)  or  J2 


or 


(2)  i  -  p(i) 


1  -  pq+lu) 


-*•  P  (i)  or  j2 


or 


(3)  i  -*•  p(l)  -*■  •  •  •  -*■  0 


^,P^+1(i)  ■*■•••  +  p^d)  -►  or  J2 


\ 


q+1 


(i)  -*•  •••  -*■  p_  (1)  -*•  j.  or  j. 


One  can  define  y^  -  £l  -  QgJ  rows  {0,  1,  n}  J2 


cols.  {0,  1,  ...»  n}  — J2>  J2 


Kl+P 

similar  to  what  was  done  in  the  V  1  -  -  case.  Then 


V0  6  “  y6Xi‘  <1  "  Ql,i  ^l5’  where  6  is  a  P°licy  °f  type  d>» 


I 


I 
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2)  or  (3).  The  computational  procedure  is  the  sane  as  before 
only  the  types  of  policies  are  more  varied.  To  get  similar 
restrictions  on  optimal  policies  here  would  take  up  twice  as  much 
space,  adding  little  of  importance.  Thus,  we  leave  this  case  as 


Rj  in  J2,  Rj  in  3^.  Rp  in  0  (some  p1p2) 


and  R 


p(i) 


in  i  (p(i)  could  be  zero),  where  p^,  p2,  p(i)  V  i  are  to  be 
determined  by  comparing  V*'s  for  various  policies. 

He  conclude  the  section  with  a  couple  of  other  special  cases 
of  interest.  Lemma  3.12  gives  conditions  under  which 


K  +p 
•*1 


'-1 


or 


Lemma  3.12:  Given  the  hypotheses  of  Theorem  3.9,  if 


A  K  +A  K  -A  K  X.K  -A  K* 

J,  jo  i  i  i  i  jn 

max  — - — ~ — ~ — -  <  min  A  — 


Ai 


i>Jl  Xjl"Xi 


then 


K  +p  K  K 

J1  J1  J2 

v  .  — or  — -  +  — - 

-1  ^4  **  *4 

J1  J1  ■'2 


no  matter  what  p  is . 


98 


1 


Proof : 


p  <  min 


AiKi"Xj1Kj1 

Vxi 


A.  <K.  +  p)  <  A.(K.  +  p)  V  i  i  j 

J1  J1  1  X 


P 


>  max 


A  K.  +A  K.  -A 
Jl  ^2  ^2 


iKi 


A  K.  +  A,  K 


j2  j2 


<  a1(k1  + 


P)  ¥  i  +  J, 


Thus,  if 


A,  K.  +A.  K  -A  K.  A  K.-X,  K4 

Ji  J,  J?  J}  1  i  i  *  ji  ii 

max  . ■ -  <  min  - —  4 


i*Jx  Ai 


i>Jl  Xjl’Ai 


^(Ki +  p)’ 1  *  h >  \\ +  p>  &  \\  +  \*l. 


V_1  is  never  Ai(KJ  +  p),  i  ^  J ^ . 


Lemma  3.12  gives  a  condition  under  which  the  V_^  optimal 

o£ 

policy  is  to  keep  either  the  component  (one  with  min  X^K^) 

working  or  the  j^8t  and  jj"**  (two  min  A^K^’s)  no  matter 
what  the  penalty  cost  is.  Some  examples  for  which  this  is  true 


(1)  X^  =  X  (Identical  component  lifetime  distributions) 

as 


min 


XiKi"Aj/j 


i  J1 


X,  -X. 


i>Jx  Jx  "1 


Xl1V'i1+X32Y'l2~Xih 


H»Ji 


K  +K  -K  < 
31  J2  1 


(11)  Suppose 


<8)  »A  -  \\  i  «i  -  V*!, * 1  *  Ji  • 

Then 


XjlKjl  "  XlK±  ~  <Xi  ~  V  KJ  V  1  * 


so 


(VVJ)^ 
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(  xl~xi\ 

K.  •  max  ll - r - )<  K.  as  X  >  X 

J2  i>‘J1  \  X±  /  J2  1  J 


j  *  1 


XiKi~Xj  Kj 


by  assumption  (8). 


So, 


<  min 


XiKi'Xj1Kj1 


<  min 


*r*i  “jj  "i 


- - - -  since  j  >  1,  X  <  X  , 

A  ,  *“  A  ,  1  1  1 


the  hypothesis  of  Lemma  3.12.  The  key  to  having  (8)  true  is  to  have  a 
sufficiently  large  spread  of  the  X^K^’s  compared  to  that  of  the  X^’s. 

In  summary,  if  the  differences  in  expected  cost/lifetimes  of 
the  components  is  large  enough  compared  to  the  differences  in  expected 
lifetimes,  the  V_^  optimal  policies  are  simpler  and  more  "intuitive". 
If,  in  addition,  the  cost/expected  lifetimes  are  ordered  in  the  same 

Kfc+P 

way  as  the  mean  lifetimes,  or  V  ■  -  has  l  small,  the  optimal 

policies  simplify. 

One  final  interesting  case  is  that  in  which 

c  ■  X.K  . XI.  Then  X,(K,  +p)>***>X(K  +p)vp>0, 

ii  nn  ll  nn 

i.e.,  if  S*  {0},  then  repair  of  the  unit  with  longest  expected  lifetime 
is  optimal.  Also,  note  that  if  p  ■  0,  the  optimal  policy  is  always  to 
let  the  system  fail  and  then  repair  that  component  with  smallest  expected 
cost/unit  time.  For  the  case  above  in  which  these  are  the  same,  we  can 
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also  say  that  the  optimal  policy  Is  to  do  nothing  In  all  states 

1,  ....  n  before  hitting  0  since  -  c.  (Of  course,  then  it 

doesn't  matter  which  component  gets  repaired.) 

For  some  examples  exhibiting  Vq  behavior  for  the  Basic 
Model,  see  Example  6.2  (Chapter  VI)  as  well  as  the  solution  to 
Example  2.1  presented  in  Chapter  II,  which  concludes  this  section  and 
chapter. 


Example  2,1  (from  Chapter  II,  Section  1) : 
Basic  Model,  n  *  2,  parallel,  L  *  0. 


Case  I  V_1  -  XjK  +  X^: 

No  transient  states  accessible  from  {12}  for  which  decisions 
must  be  made. 


12  0 

optimal  policy  Rj  R^  -  ^  ■  (1,  2}  . 


Case  II  V_1  -  X^l^  +  p). 

If  X  h  >  X^K2  (^  *  2),  then  by  Case  I. A.  (Application  of 
Lemma  3.11), 


12  0 

optimal  policy  A  A  R^^  ■  {0}  . 


If  Xj^  <  X2K2  (j^  ■  1),  then  by  Case  I.B.  (Lemma  3.11 
Applications)  the  optimal  policy  is: 
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12  0 
A  A  Rx  , 


/  U2  \K1 
-1  \  \!  ^ 


/  {0}  . 

1  2 

0 

/ 

Ki  1 

A 

R1  • 

if  v-i  ’  I1 + 

i) 

Case  III 

II 

iH 

1 

> 

X2(K2  +  p): 

If  A^ 

<  X2K2 

(jt  =  1).  then  by 

Case 

II. A.  of  Lemma  3.11 

(Applications) , 

1  2 

0 

optimal  policy  A  A 

R2 

<?=  {0}  . 

If  AjK- 

>  a2k2 

(j1  =  2),  then  by 

Case 

II. B.  of  Lemma  3.11 

applications,  the  optimal  policy  is: 

1  2 

0 

/ 

\  k2 

A  A 

r.  » 

if  V  <  1  1 

+  -i 

2  ’ 

-1  \ 

U2, 

hi 

1  2 

0 

/ 

W,  ' 

\  k2 

R„  A 

R„  , 

if  V  ,  >  1  1 

+ 

—  . 

2 

2  ’ 

-1  \ 

>Y 

hi 
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CHAPTER  IV 


THE  DEGRADATION  MODEL 


1.  Description  of  Model 

In  the  Basic  Model  it  was  assumed  that  system  components 
are  either  working  or  failed  (on/off)  with  exponential  lifetime 
distributions.  However,  for  many  systems,  the  components  may 
be  observed  in  some  finite  number  of  states  of  increasing 
degradation  before  failure.  The  more  degraded  the  component  at 
the  time  of  repair,  the  greater  the  repair  cost.  The  Degradation 
Model  takes  this  factor  into  account ,  having  the  following 
Coherent  System  Repair  Model  parameters: 

States:  each  component  can  be  in  one  of  "£"  states  of 
degradation  or  new. 

nth 

0  =  new 

*  failed 

1,  ...,  I  -  1  =  degraded  (not  failed) 

Repair :  instantaneous,  unlimited  service  -  brings  degraded 
component  to  "new"  condition 

Component  Lifetimes:  Let  Lk  ■  lifetime  of  itk  component 
given  it  is  in  the  kC^  degradation  state,  0  <_  k  < 

-Xkt 

k  c  k 

The  L^  ~  1  -  e  •  p{L*  t}. 
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r 


s  t 

Notice  that  if  £  «  1  (1  degradation  state  *  failed), 

then  the  Degradation  Model  reduces  to  the  Basic  Model. 

Of  course,  in  the  process  of  increasing  the  number  of 
states  in  the  model,  a  price  must  be  paid  in  the  difficulty  of 
obtaining  results.  Only  the  general  series  case  (series  systems 
are  viable  now  unless  £  =  1)  and  the  identical  component 
parallel  cases  are  treated,  and  for  V  optimality  only.  Luckily, 
in  the  series  case,  V_^  optimality  is  sufficient  to  give  the 
general  optimal  solution  and  in  the  parallel  case  it  gives  the  key 
information  desired.  In  utilizing  this  model,  it  would  certainly 
be  advantageous  to  keep  the  number  of  degradation  states  as  low 
as  possible,  while  still  capturing  the  essence  of  the  system  being 
modeled,  as  the  total  number  of  states  is  of  the  order  (£  +  l)n 
where 


£  =  number  of  degradation  states 
n  =  number  of  components. 

Section  2  treats  the  general  series  case  while  Section  3  looks 
at  the  parallel  case  in  identical  components  with  no  fixed  charge. 
Section  4  presents  some  possible  changes  in  state  and/or  decision 
space  in  this  model  to  provide  extensions  of  the  Basic  Model  to 
Erlang  component  lifetimes  as  well  as  provide  insight  into  other 
possible  non-constant  failure  rates  for  components. 

The  number  of  parameters  needed  to  specify  the  model  is  in- 


increased  k-fold  from  the  basic  case  and  are  as  follows: 


(1)  fixed  charge,  L  _>  0, 

(2)  penalty  cost  for  system  failure,  p  >  0, 

(3)  vu  =  1/X^  =  expected  length  of  time  iC^  component 

spends  in  the  degradation  state  (exponential) . 

These  are  defined  for  k  =  0,  1,...,£-1  and  i  =  l,...,n. 
k  =  i  is  not  defined  since  component  failed  there. 

(4)  *  cost  to  repair  ith  component  when  in  the  kC^ 
degradation  state. 

These  are  defined  for  i  =  1,  ...,  n  and  k  =  1,  . . . ,  £ 
(k  =  0  is  new  and  no  need  to  repair  there). 

12  £ 

0  <  <  •  •  •  <  is  assumed  to  allow  for 

increased  costs  to  repair  in  higher  ("more")  degraded  states, 
v  i  =  1,  . . . ,  n. 

A  state  in  this  model  is  denoted  by  a  vector,  s,  of  length  n; 
s  =  (sp  s^,  ...,  sn  )  where  *  k  if  component  i  is  in  the  kth 

state  of  degradation.  A  change  of  state  occurs  when  one  of  the 
components  enters  a  higher  state  of  degradation  and  at  such  an 
instant,  a  decision  is  made  to  repair  some  subset  of  the  degraded 
components  or  to  do  nothing,  (some  repair  need  only  be  done  if  the 
system  is  down) .  Repair  is  always  assumed  to  bring  a  component  back 
to  a  "new"  condition.  Each  state  as  defined  now  has  exponential 
holding  time  so  the  Degradation  Model  is  also  a  continuous  time 
Markov  decision  chain  with  infinite  time  horizon. 
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As  before,  specifying 

(1)  state  space 

(2)  decision  space 

(3)  transition  structure 

(4)  cost  structure 

(5)  objective  function 
defines  the  model  completely. 

The  Basic  Model  notation  in  Figure  2.1  applies  here  except  that 
the  component  related  cost  and  mean  lifetime  parameters  now 
depend  on  degradation  state  (superscript)  as  well  as  the  component 
(subscript).  Also  let  l  =  number  of  degradation  states.  Obviously, 
these  could  vary  by  component  but  for  notational  simplicity,  1 
assume  they  are  the  same  for  all  components. 

The  Markov  chain  specified  by  the  Degradation  Model  is  as 
follows: 

State  space:  States  =  which  components  in  what  degradation 

states  s  =  <  s, ,  . . . ,  s  >  .  If  component  i  is  in  the  s.1'*1 
'■in'  i 

degradation  state,  0  <_  s^  £  l.  If  system  is  k-of-n  and 
components  are  identical,  then  states  are  the  number  of 
components  in  various  degradation  states,  i.e., 

?  =  ^  s0*  si*  sp^  >  where  s^  components  are  in  the  it^' 

degradation  state,  s^  +  s^  +  • • •  +  s^  =  n. 
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Decision  Space:  Let  ft  -  {1,  ...»  n}.  Possible  decisions 

in  state  s  =  < s^,  sn>  are  R^  where 

s 

8s  C  il  -  (i  :  s^  »  0)  =  set  of  non-new  components  in  g. 

-  Qg  *  $  *  A  (do  nothing) 

s 

-  if  system  is  down  in  s,  then  ¥  <P  • 

s 

Transition  Structure:  The  transition  matrix,  Qq  :  (assuming 

6  :  R^  in  s) .  Fix  s  «  (s^,  S2»  sn>  . 

§ 

Definition:  State  t*  *  ( t . t,  .  ,  t.+l.t . t  i 

-  -  1*  '  i-1  i  ’  i+1*  ’  n' 

if  t  =  ( t^,  t^,  ....  tR)  (t  is  t  with  il  component  in  a 

higher  degradation  state),  t*  is  only  defined  for  i ] t±  <  l 

(non-falled  components).  Also,  let  state  t  U  !J.  ■  ( t?",  ...,  t1) 

,  c  x  n 

where 


f>. 


if  i  e  n 
if  i  ^  nt 


Note  t__UJi  is  the  condition 
of  the  system  after  instantaneous 
repairs  in  state  t. 
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<v 


s.(skiS3) 


sh 


n  (sUJl  ). 
'8  j 


l  Y 

j-i 3 


otherwise 


_1 _ 

%?s>j  ' 

.1 


Cost  Structure: 


if  system  up  in  s 


if  system  down  in  s 


where 


if  n.  -  ♦ 
otherwise 


Objective  Function: 


*  5 
P6R0 

*  6 
Vi 


(any  possible  optimal  6  will  form  a 
Markov  chain  with  a  single  irreducible 
set  of  ergodic  states). 
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Belov  are  examples  of  Che  series  and  parallel  cases: 


Example  4.1:  n  -  2,  i.  -  2,  series  system,  L  -  0 

Then,  possible  component  states  are: 

0  ■  new 

1  •  1st  degraded  state 

2  -  2nd  degraded  state  (failed). 


system  up  system  down 


States:  1 

00 

01 

10 

11  " 

02 

20 

12 

21 

Decisions: 

— 

A,R2 

a,r2 

A  i  f  R2 » 

R2 

R1 

R2 

R1 

CM 

Cost : 

0 

O.Kj 

O.kJ 

O.K^.K* 

K2 

R2 

K2 

*1 

K2 

K2 

K2 

K1 

Note:  Since  series  model,  there  is  no  need  for  p  because 

2 

system  failure  *  component  failure  **  can  include  p  in  K^'s. 


Sample  of  Qn:  Let  6  :  A  in  01,  R,  in  10,  R,  in  11 


00 

01 

10 

11 

02 

20 

12 

- 

-x° 

,0 

00 

1 

X2 

X1 

n 

0 

A 

x°+x° 

12 

>°+x° 

l  2 

u 

0 

0 

_xo 

_X1 

01 

1 

0 

X1 

X2 

0 

A 

\J 

x^+xx 

12 

x^+xx 

ArA2 

0 

_x° 

x0 

10 

0 

A2 

X2 

0 

0 

A 

A 

x°+x° 

1  2 

x°+x° 

VA2 

0 

0 

X1 

_xi 

11 

n 

a 

0 

X2 

X2 

0 

A 

u 

xX 

x^x1 

12 

0 

_x° 

_xo 

02 

X2 

X1 

A 

u 

0 

1 

0 

0 

20 

0 

A 

r\ 

0 

A 

Xl+X2 

X°+A0 

Ar  A2 

U 

1 

0 

-x° 

X1 

12 

n 

0 

0 

X2 

0 

"X1 

w 

x^+x® 
A1  A2 

X^X0 

1  2 

1 

_X° 

_X1 

21 

n 

n 

X1 

X2 

0 

n 

u 

X°+xX 
A1  A2 

x°+xx 

A1  A2 

U 

Example  4 

.2:  n 

-  2,  l  - 

2,  L  - 

0,  parallel  system  with 

identical  components. 


Ill 


f 


system 

system  up  down 


States:  '200 

110 

101 

020  011  1 

*  002  1 

Decisions:  - 

A,R^ 

*,^2) 

A  IL(1)  R(1)  A  R(1>  R(2> 

9  **2  *»  f  |  | 

r(1)+(2) 

R1 

»(2)  „ (2) 
l  ,R2 

Cost:  0 

0.K1 

0,K2 

O.K1^1  0,K1,K2,K1+K2 

k2+p, 

2K2+p 

where  ^  denotes  repair  of  i  identical  units  which  are  in  the 
jth  degradation  state. 

Note  the  p  is  now  necessary  to  distinguish  between  component  and 
system  failures. 


2.  The  Series  Case 

Consider  now  the  case  of  a  series  system,  i.e.,  one  which 
fails  as  soon  as  any  one  of  its  components  does.  As  long  as  no 
component  has  reached  the  l  degradation  state  (has  failed), 

decisions  can  be  made  to  repair  any  subset  of  the  degraded 
components.  If  the  system  fails  due  to  a  failure  of  component  1, 
then  the  required  decision  is  to  immediately  repair  component  i 
(no  matter  what  condition  the  others  are  in). 


The  following  theorem  gives  the  optimal  policy: 


Theorem  4.1:  We  have  the  Degradation  Model  with  a  series 


system: 


Let 


*1 

j  *  argmin  - 

l<J<t  '  p 
L  Wi 
P-1 


Then  the  optimal  policy  is  one  which  does  nothing  to  component  1 
until  it  reaches  the  state  of  degradation  and  then  repairs  it. 

Proof:  Directly  from  Lemmas  4.2,  4.3  which  follow: 


Lemma  4.2:  Suppose  we  have  the  series  Degradation  Model. 
Fix  a  policy  6.  Then 


,  n  *  ,  it; 

"  i  i  •  jTi —  > 
‘-1  J‘1 

P-0  1 


i  j 

where  £  or  “  1 ,  v  i  »  1 ,  ...,n  and  a ^  _>  0 

j-1  1 


Proof :  Fix  6 . 

* 

Let  P  -  stationary  transition  probability  vector  given  6 
-  <P>8)  for  possible  states  s. 

Then  P*(l  -  qQ)  -  0  . 

(Since  6  is  fixed,  we  drop  it  to  simplify  notation). 


fc. 
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1 

i  n 

* 

* 

l  l  N  l 

» 

pQl 

pQl 

j-l  i-l  1  s:R1 

In  s 

and 

vJ 

( 8  *  (  •  •  •  9  8^)) 

recall  *  degradation 
state  of  i  in  s  * 


u  , 

-  I  l  «1 

j-l  i-l 


*1 


p-0 


P 

i 


where 


It  remains  to  show 


1.  Without  loss  of  generality,  set  i-l. 


To  show 


1,  it  is  sufficient  to  show 


Some  notation  is  now  introduced  for  use  in  this  proof: 
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* 


f  t 

Definition:  Fix  <5.  Let  <9t  *  F  P  _  be  the  sum 

-  *  t|t6nt-.  •* 

of  stationary  probabilities  of  all  states  t  which  are  sent  to  s 
instantaneously  under  6. 

-  Notice  that  9?  is  only  defined  for  s  such  that  system  is  working 
s 

since  if  the  system  is  failed  in  s,  ^  t|t  U  11  ■  s  by  model  definition. 


i+  i+ 

Definition:  Let  s  (s  )  be  the  state  for  which  all 
components  are  in  the  same  degradation  state  as  in  s  except  for  the 
iC^,  which  is  in  one  lower  (higher)  state  of  degradation.  If 
s^  *  0,  then  si+  is  not  defined. 

From  the  definition  of  9ft 


(2) 


P 

•s 


n 

l 

i-1 

i:s^0  "i 


A-1 


s.-l  n  s 
l.1  +  £a 
j=l  ^ 
J*i 


j 


9Tii  . 
s 


Thus, 


(3)  W3 


t:tun  =s 


l 

t:tun 


t  ? 


n 

l 

i=l 


V1 


t.-l 


i|t^0  "i 


1  %  l  ^ 

j-i J 
J*. 


Let  9t  m  ( ),  s  |  system  up. 

Let  R  “  ((R  _))  _  ,  where 

'  st  s ,  t 
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n  t , 


if 


t1*  u  n 


St 


l  xjj 

j-1 3 


i+ 


,th 


(i.e.,  if  i'~“  component  of  t  \ 
degrades,  go  to  s  immediately] 
under  6  / 


otherwise 


Restating  (2)  in  matrix  form  gives: 


(I  -  R)  0  . 


If  R  *  sth  row  of  R  and 
§• 

lg  =  vector  of  zeros  except  1  in  the  s^  spot,  then 


(A)  (1  -  R  m)  0,  v  states  s>  system  is  up. 


This  is  just  a  restatement  of  (3)  in  vector  form. 

»s 

To  prove  (1) ,  one  must  get  everything  in  terms  of  W  , 

s 

rearrange  terms  and  using  certain  sums  of  equations  (4),  show  it 
to  be  zero. 


Now, 


Thus 
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(1) 


* 

p  Q, 


i  Ci  -7)  i  *.t 

j=l\p-0  Aj/tsRj.  in  t 


#1  in  j 


1-2 
I  ~ 

q=0  X« 


n  s 

lx  j 

J-l  3 


(q,s) 


1-2 

l 

k*q+l 


s|r[  in  P*(k^> 
(k,s) 


where  s  is  a  configuration  of  components  2,  ....  n  =  (s2,  . ..,  s^) 
and  state  (q,  s)  =  < q,  s2,  ....  sn>  by  (2), 


,  k-1 


(k,s) 


..ns 

^  +  l  X3 

j-2  2 


j  +  i=2 


n 

I 


,'f1 


v1 

i|s^oXi  +x,  j=2 


k+  y  x j 

1  j-2  j 
j1*i 


^  it 

(k,slf) 


Substitution  of  this  into  the  previous  expression  for  (1)  =* 


(1) 


1-2 


I 

q=0 


R(q,s) 


)  9? 


-  0  by  (4)  . 


Note:  The  proof  is  valid  for  any  decisions  in  working 
states,  but  assumes  that  in  states  where  an  item  is  failed,  repair 
of  only  the  failed  item  will  take  place,  not  an  unreasonable  assumption. 
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Lemma  4.3  now  identifies  the  policies  which  correspond  to 
* 

1  1  * 

extreme  points  (a^  =  1,  an  =  0,  jfj)  in  the  convex  set  of 
*  s 

V  for  possible  policies  in  the  Degradation  Series  Model. 


Lemma  4.3:  Suppose  we  have  the  Series  Degradation  Model 
and  let  be  defined  as  in  Lemma  4.2.  Then  a  policy  6  has 


‘“i1  =  1 


=  0,  j  5^  3± 


<*  6  repairs  the  ith  component  whenever  it  reaches  the  j  ^  state 
of  degradation. 


Proof : 


Suppose  6  repairs  the  ith  component  whenever  it  reaches 
the  state  of  degradation,  leaves  it  alone  otherwise. 


£  n  .  K, 

V_1  "  1  l  “J  jIT 

p*0 


in 

J  =  1JP.--0  V 


l  p.„  • 


where  a .  =  . 

i  *  u  •  s 

p  siR^  in  s 


Since  ^s|r^  in  s  if  i  is  not  in  j  and  P#g  *  0 
vs|si  >  j ^ ,  a|  =  0  for  j  +  ji  and  since 
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by  lemma  4.2, 


=  1. 


I 

l 


.1*=! 


□ 


(j*)  Pick  i.  Suppose  =  1,  a|  =  0,  ]  ^  j^.  aj  =  0 

=*  P  =  0  V  s}R,  in  s  and  1  in  j  #  j  .  and  s  is  in  the 
•si  1 

ergodic  chain  entered  by  starting  in  the  state  with  all  components 
working . 


=>  No  such  s,  I.e.,  6  leaves  component  i  alone  unless 
in  state  of  degradation  j^.  □ 

These  two  lemmas  immediately  prove  Theorem  4.1: 


Proof :  (Theorem  4.1) 

Let  V  °pt  =  min  V6  .  Fix  5  . 
-i  6  -l 


Lemma  4 . 2 


n  i 

l  l 

i=l  j=l 


i-l  n 

lA 

p=0 


1  I 


i=l 


C 


i 


min 

i£j^i 


by  convexity. 


But  I  C  =  V°1  where  o  is  the  hypothesized  optimal  policy.  □ 
i=l  1  -1 
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Note:  No  assumptions  on  orderings  of  the  A^'s  with  respect 

1  J/, 

to  components  i  or  j  are  needed.  Also,  the  <  •••  <  ■v  i 
is  not  needed  but  is  added  since  if  k|  >  some  m  ^  1,  then 

servicing  i  in  degradation  state  j  is  never  optimal. 

Note :  Theorem  2.7  holds  trivially  here,  i.e.,  series  =  n  of  n 
system  and  never  repair  until  n  -  1  items  are  left  working. 

To  clarify  some  of  the  procedures  and  proofs  in  this  section, 
consider  Example  4.1  of  Section  4.1. 


Example  4.1  (continued) : 

series  system 

n  =  2  (2  components)  SL  =  2  (0,  1,  2  ■  degradation  states) 

_ system  up _  down 

states:  R)0  01  10  11  I  f  02  20  12  21-1 

automatic  decisions 

Let  6  -  A  R  R  R  R  R  R 

(as  before)  11  till 

Using  the  notation  of  the  proof  of  Lemma  4.2, 
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121 


122 


Set  of  policies  which  could  possibly  be  optimal: 


Component 


11 _  #2 


Correspond. 

Decisions,  a 

6  Sm 

01  10  11  {ergodic  states} 


repair  in 
degradation 
state  1 


repair  in 
state  1  2  1 


{10,01} 


k: 


K2 


0  0^  1 
W1  V2+U2 


repair  in 
state  1 


repair  in 

state  2  A  R  R  {10,01,11,02} 
(failed) 


^  +  - ~ 


o .  1 

vl+vl 


repair  in 

state  2  i  R9  A  R„  {10,01,11,20} 

(failed)  8tate  1  1  L 


2  2 
h  +  K2 


0.1  ° .  1 

VU1  Vu2 


repair  in 
state  2 
(failed) 


repair  in  { 

state  2  AAA 
(failed) 


K„ 


Suppose  C^  =  — ^ 


0 ,  1  * 
p2+h2 


Then  the  optimal  policy  would  be 


01  10  11 

A  R^  by  Theorem  4.1. 


Note: 

(1)  Given  the  general  form  of  an  optimal  policy  in  this  case, 
it  is  clear  that  one  never  repairs  more  than  one  item  at  a  time  in 


any  state  since  only  one  Item  at  a  time  can  change  degradation  state 
and  repair  Is  done  Immediately  and  instantaneously  upon  entry  into 
the  "key"  state  for  repair. 

(2)  If  the  components  are  identical,  then  the  optimal  policy 
is  to  repair  whenever  any  component  reaches  a  certain  degradation 
level 


d  =  argmin  v  i 

l_<j<ll  P 

l  »  j  1 

p=0  =  pJ  V  j  . 

(3)  Theorem  4.1  is  so  intuitive,  there  ought  to  be  an  easier 

way  to  prove  it,  i.e.,  without  having  to  use  the  stationary 

* 

probability  equations  p  (I  -  Qq)  *  0,  using  only  independence, 
exponentiality,  and  series  structure.  If  such  a  proof  could  be 
developed,  one  should  then  easily  be  able  to  show  Theorem  4.1 
is  true  for  the  case  of  different  numbers  of  degradation  states  per 
component.  I  conjecture  that  the  result  also  holds  for  "blocks"  of 
components  in  series,  [reference  Chapter  VI  for  Basic  Model  case]. 

3.  The  Parallel  Case,  Identical  Components 

This  section  treats  the  Degradation  Model  for  the  case  of  a 
parallel  system.  Identical  components  are  assumed  to  simplify  the 
system  to  the  point  where  general  results  could  be  obtained  for 
V using  theorems  similar  to  those  encountered  in  solving  the 
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Basic  Model.  These  state  that,  since  we  have  a  parallel  system  and 
instantaneous  repair, 

(i)  one  will  never  repair  more  than  a  single  unit  at  a 
time  in  an  ergodic  state  in  a  V_^  optimal  policy 
(ii)  one  will  never  repair  until  the  system  gets  down  to 
a  single  unit  working. 


Showing  these  two  results  would  limit  the  number  of  possible 
V optimal  policies  to  a  manageable  size.  Precise  V_^ 
results  will  be  presented  in  some  examples  for  £  =  2  and  £  *■  3, 

(£  3  number  of  degradation  states)  and  compared  to  Basic  Model  results. 


In  the  Basic  Model  we  were  able  to  state  Theorem  2.1  which 

6  Yi 

expressed  V_^  for  any  possible  6  as  a  convex  combination  of  V 

where  y^'s  are  policies  which  involve  repair  of  at  most  one  unit  at 

a  time  in  any  state.  For  the  parallel  case  with  different  components 
Yi 

these  V_^  were  such  that 


or,  for  identical  components, 


rYi  ]k+£  2K  3K  nK 

-1  \  W  *  li  '  M  *  '  •*  V  ’ 
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Given  no  fixed  charge,  Che  V  optimal  policy  was 
extremely  simple:  do  nothing  until  one  component  is  left  working 
and  then,  if  the  penalty  cost  is  large  enough,  repair  one  unit 
(6^)  (continuing  this  policy  on  forever)  or  if  p  is  small  enough, 
let  the  system  fail  and  then  repair  (6q)  one  unit  ad  infinitum. 
Under  6  ,  £ •  set  of  ergodic  states  =  (1)  , 
under  6q,  £  *  set  of  ergodic  states  -  {0}  , 
where  state  i  indicates  the  number  of  working  components. 

In  the  series  case  of  the  Degradation  Model,  we  were  able  to 
eliminate  most  policies  6  from  being  V  optimal  by  again 


convex 


6  6  6 
expressing  V for  any  6  as  \  ,  where  the  C°  are 

1  1=1 
Ki 

combinations  of  -r— j —  .  An  optimal  policy  was  found  by  computing 

K 

p-0 


„opt 


K. 


min 

i<i±*  -j  „p 

p-0 


—  for  1  i  <_  n,  the  optimal  policy  being  to 


tth 


ri 


repair  component  i  whenever  it  reaches  the  j. 


argmin  j- r - 

l£l<t  JV  yP 

p-0  1 


state  of  degradation.  In  the  identical  component  case,  this 


.th 


corresponds  to  repairing  any  component  that  reaches  the  j 

,1 


degradation  state,  j  *  argmin  r- 

^  iV 

p-0 
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Unfortunately,  In  the  parallel  case  as  is  exhibited  in  the 


example  below,  there  are  no  such  "nice"  convex  combinations  of 
simpler  policies  that  can  be  written  as  for  any  6.  The  series 

result  depended  heavily  on  the  fact  that  when  the  system  (and,  thus, 
any  component)  failed,  repair  of  the  failed  component  had  to  be  done 
immediately,  a  fact  obviously  not  true  for  a  parallel  system.  In 
this  case,  even  if  p  =  0,  V  1 V  <5  is  not  a  convex  combination  of 


K 
-1 
l  MP 


or  a  sum  of  such  combinations.  However,  the  relationship 


p=0 

between  them  still  appears  to  be  the  principal  factor  in  determining 
at  what  degradation  state  to  repair  a  component. 

Since  the  model  is  parallel,  repair  is  instantaneous  and  there 
is  no  fixed  charge,  one  intuitively  expects  that  as  with  the  Basic 
Model  that  repair  will  never  be  undertaken  on  more  than  one  unit  at 
a  time  and  repair  will  never  be  done  until  there  is  only  one  unit 
left  working.  These  results,  though  true  for  the  Basic  Model  identical 
component  case  are  much  more  difficult  to  prove  given  failure  of  the 
Basic  Model  theorems  or  likenesses  thereof  and  the  increased  complexity 
of  model  structure  in  the  Degradation  case.  I  conjecture  these  two 
results  are  true  even  if  the  components  are  different  but  leave  it 
as  a  topic  for  future  research  (see  Chapter  VII) . 


1 


Example  4.2:  n  *  2,  l  *  2,  L  *■  0,  parallel  system  with 
Identical  components. 


States:  200 


_ system  up _ 

110  020  101 


Oil 


system 

down 

002 


Decisions: 


A,R 


(1) 


A,R 


(1) 


1 

(1) 


A  B(2>  A  d 

A  f  R«  A,R. 


(1)  r(2)  „(2)  d(2) 


r; 


,(1)&(2) 


The  optimal  policy  will  be  found  by  enumeration  of  all  the 
17  possible  different  V  ^'s.  Table  4.1  lists  the  possibilities,  the 
set  of  ergodic  states  induced,  and  the  V_^'s. 

Let 

K*  ■  cost  to  repair  a  component  in  ith  degradation  state 

*  expected  length  of  time  spent  in  the  ith  degradation 
state  by  a  component. 


128 


TABLE  A.  1  -  Effimt  RATIO*  Of  POLICIES  EOlt  EXAMPLE  A.2 

_ t _ 

ypiicv  wo.  1  no  oao  101  on  "ott"1  &  -  i.ruodic 

*  >  eJ11  -  <on) 

*  *  -  -  -  A  R*2)  (011,002) 

*  3  -  A  A  IjJ)  -  (020,101,011) 

O)  *  -  Ej”  A  R<2)  -  (020,101,011) 

(23)  5  A  A  A  A  R<2>  (1)0,020,101,011,002) 

(1)  0  A  R*1’  A  R<2>  -  (110,020,101,011) 

(1)  7  A  A  R<2>  Rj<2)  -  (110,020,101,011) 

U)  8  A  r”’  R<2)  -  -  (110.020.101) 

(13)  9  A  A  A  R<‘><2>  -  (110, 020, 101,011) 

<1)  10  *  RjU  R{2)  -  -  (110.020.101) 

(123)  11  A  R*1’  A  A  R<2>  (110,020,101,011,002) 

(1)  12  *<”  -  -  -  (110) 

(13)  13  A  A  R<2>  *<*>«>  -  (110,020,101,011) 

(1)  14  *  ■  *”  A  pd)(2)  .  (110,020,101,011  ) 

(D  15  A  R<U  A  RU,<2)  -  (110,  020,  101.  Oil) 

(12)  *6  *  rJ1*  A  A  «<2>  (110,020,101.011,002) 

,2,)  W  A  A  r|2)  A  R*2)  '110,020,101,011,002) 

*  -  possible  opt  Isa]  policy. 

(i)  •  loses  to  policy  i. 

(ijk)  •  loses  to  1,,  or  k. 


The  following  facts/patterns  appear  from  Table  4.1: 

(1)  The  optimal  policies  are  either: 

a.  Do  nothing  until  one  left  working  in  first 
degradation  state,  then  repair  it. 

b.  Do  nothing  until  one  left  working  in  first 
degradation  state  and  then  repair  the  failed  unit. 

c.  Do  nothing  until  system  fails,  then  repair  a 
component. 

(2)  Never  repair  until  one  component  left  working  and  never 
repair  more  than  one  unit  at  a  time. 

(3)  Uq  divides  denominator  of  **  do  not  need  policy  3 

to  dominate  6. 

(4)  No  "p"  =*  do  not  need  2  to  dominate  6. 

(5)  No  "K1"  i.e.,  never  repair  in  first  degradation  state 

do  not  need  policy  1. 

(6)  Although  the  optimal  policies  are  simple  and  intuitive, 
the  Vj's  for  other  6  are  not  convex  combinations 
of  these  policies.  They  are,  however,  _>  some  convex 
combinations  of  these  policies. 

Given  the  results  of  Example  4.2  and  others  tested,  three 
facts  appear  to  be  true  in  the  identical  component  case  and  probably 
in  general  for  a  parallel  system.  These  are: 

(1)  Never  repair  more  than  one  unit  at  a  time  in  an  ergodic 
state. 


130 


(2)  Never  repair  until  there  is  one  unit  or  less  working. 

(3)  The  V  ^  optimal  policy  for  a  parallel  degradation  model 
with  n  identical  components  and  i  states  of  degradation 
is  the  same  as  that  for  a  2-component  model  with  l 
states  of  degradation. 

Proofs  of  these  facts  using  Degradation  Model  stationary 
probability  equations  to  eliminate  certain  policies  as  convex 
combinations  of  others  as  in  the  Basic  Model  case  are  nearly  impossible, 
due  to  the  greatly  increased  complexity  of  their  structure.  They  are 
left  as  a  topic  for  future  research  (Chapter  VII). 


4.  Extension  of  Basic  Model  to  Include  Erlang  Component  Lifetimes 

The  purpose  of  this  section  is  to  describe  how  the  Degradation 
Model  can  be  used  to  extend  the  Basic  Model  to  the  case  where 
component  lifetimes  are  no  longer  exponential,  but  Erlang.  An  example 
will  be  solved  of  such  a  model  where  components  are  non-exponential. 

Its  solution  will  demonstrate  that  changing  component  lifetime 
distributions  does  change  results  given  for  exponential  -  in  particular 
it  is  no  longer  true  that,  for  no  fixed  charge,  one  never  repairs 
more  than  one  unit  simultaneously. 

Suppose  now  that  we  have  the  Basic  Model  with  L  *  0  except 
that  L^,  the  random  variable  representing  the  lifetime  of  the  i^ 
component,  has  an  Erlang  distribution,  i.e., 
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t  n  n-1 

p{ L .  <  t}  =  /  j -2fr,  e  Z  dz  =  G  (t) 

i  —  q  (n-1) !  A ,n 


n-1  .  i 

,  r  (At)  -At  ,  ,  _ 

=1-2  — n e  ,  n  >  1,  an  integer,  A  >  0 

1=0  1  ' 


Definition:  (t)  is  a  gamma  distribution  with  parameters 

““ A  y  n 

A,  n,  where  A,  n  >  0.  If  n  is  an  integer  >  1,  then  G,  (t)  is 

A  y  n 

called  an  Erlang  distribution.  If  n  =  1,  then  G,  , (t)  is 

A  ,  i 

exponential. 

If  n  1,  G  (t)  is  a  DFR  (decreasing  failure  rate) 

A  9  n 

distribution  and  if  n  _>  1,  it  is  IFR  (increasing  failure  rate)  where 

Definition:  Suppose  F(t)  is  a  probability  distribution 
function  with  density  f(t).  Then  the  failure  rate  at  time  t  at  a 
unit  whose  lifetime  distribution  is  F(t)  is  r(t)  =  f(t)/l-F(t). 

Intuitively,  this  is  the  rate  of  change  of  probability  of  unit 

failure  at  time  t.  (DFR) /IFR  indicates  a  (lesser)/greater  chance 

of  failure  with  age.  Exponential  distributions  have  a  constant 

failure  rate,  i.e.,  the  chance  of  failure  is  independent  of  how  long 

the  component  has  been  working. 

Up  to  now,  the  Basic  Model  has  assumed  components  with 

constant  failure  rate.  Components  having  G  distribution  allow 

A  j  n 

modeling  of  systems  where  components  might  have  lFR(for  n  1) 

distributions.  Figure  4.2  indicates  the  failure  rate  curves  for 

G,  . ;  G,  „  and  G,  .  for  A  =  1. 

A ,  1  A  , 2  A  ,4 
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FIGURE  4.2  -  FAILURE  RATE  CURVES  FOR  GAMMA 
DISTRIBUTION  FOR  A  =  1. 


Notice  that  as  n  increases,  the  probability  density  for 


G^  (t)  becomes  more  peaked,  i.e.,  the  variance  of  the  lifetime 
A,n 

from  its  expected  value  gets  smaller  as  n  gets  larger.  Since 


G,  (t)  has  a  mean  (expected)  value  of  1/nA,  the  distributions 
a  ,  n 

G,  .  (t)  and  G,  ,  (t)  =  1  -  e  ^  will  have  the  same  mean,  1/A 

A/n,n  A ) 1 

but  G,  ,  (t)  will  have  a  much  lower  variance  (more  confident  of 

A  /n  y  n 

expected  value) . 


Recall  from  elementary  probability  distribution  theory  that 


if  a  random  variable  X  ~  G  (t)  [p(X  <  t)  =  G  ( t ) ] ,  then  X  can  be 

A ,n  —  a  ,n 

written  as  a  sum  of  n  exponential  random  variables,  each  with 


r  -At 

mean  1/A.  Thus,  X  =  £  X  where  X  ~  1  -  e  .  It  is  this  idea 

i=l  1  i 
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which  allows  the  use  of  the  Degradation  Model  in  extending  the  Basic 

Model  to  the  Erlang  case  by  allowing  components  to  pass  through  n 

"fictitious"  states  of  degradation  before  failure.  Of  course,  unlike 

the  Degradation  Model  in  which  these  intermediate  states  can  be 

observed  and  actions  taken  in  them,  decisions  here  are  still  limited 

to  the  instant  of  component  failures  since  the  intermediate  states 

do  not  really  exist.  Thus,  if  we  want  to  extend  the  Basic  Model  to 

the  case  where  the  n  independent  components  have  lifetimes 

G.  (t)  =  p{L.  t},  then  a  Degradation  Model  for  n  components 
A  ,  ,k.  1 

1  1 

can  be  used  with  the  following  modifications: 

(1)  Allow  k^  degradation  states  for  component  i, 

t  h 

(k^  =  failure)  each  with  mean  holding  time  1/X^. 

(2)  Restrict  repair  decisions  to  times  when  a  component  fails 
(enters  state  k^) ,  otherwise  do  nothing  (use  same 
decision  in  states  with  same  configuration  of  failed 
components) . 

(3)  Component  i  has  mean  life  1/k^X^  here.  To  get 
desired  mean  life,  can  adjust  A  (A^  =  A/k^  gives 
mean  1/  A  ) . 

Given  the  limitations  on  decisions,  the  theorems  for 
Degradation  Model  solutions  will  not  apply  here  but  the  model  as 
stated  is  a  Markov  decision  chain  and  can  be  solved  as  such  either 
by  hand  or  on  the  computer. 

To  illustrate,  consider  the  following  simple  example: 
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Example  4.3.  Basic  Model,  n  =  2,  L  =  0,  parallel  system. 


Ll  ~  G(u1/2,  2) 


mean  *  1/p 


“V 

L2  ~  G(u2»  1)  =  1  -  e  mean  =  1/p, 


be  Mj,  the  extended  Basic  Model. 


L  ~  1  -  e 


mean  =  1/p 


L2  ~  1  -  e 


mean  =  1/p, 


be  M2,  the  standard  Basic  Model. 


M2  has  states:  12 


(Basic  Model 
Notation) 


decisions: 


has  states: 


both  up  (12) 

(00  10  01  D 


R1,R2,R12 


It  1  up  (1)  It2  up  (2)  failed  (0) 

f02  12 1  120  21*  I  22  I 


decisions: 


same  decn. 
applied  to 
both 
02,  12 


A,  Rx  \  Rr  R,, 

same  decn. I  R^2 

applied  to/ 
both  1 
20,  21  1 


The  set  of  possible  policies  is  the  same  for  both  and  M2> 

Below,  in  Table  4.3  is  a  list  of  possible  policies  and  corresponding 


V  ^'s  under  and  M2> 
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TABLE  4 . 3  -  TABLE  OF  V_1's  FOR  AND  M2  OF  EXAMPLE  4.3. 


Policy 
12  0 


£ 


M, 


{0} 


M„ 


{22,12} 


M, 


M„ 


K1+P 


{0} 


{22,21} 


k2+p 


K2+p 


R2  R1 


{1,2}  {10,01,11 ,02 
20,12,  21} 


Mi  M2 


AAR 


12 


{0,1,2}  {22,10,01,11, 
02,12,20,21} 


(m1+u2)(k1+k2+p) 

2  +  2+2"2l(“2+T) 

“l+ul"2+u2+-  „1+„2 


(m1+m2)(k1+k2+p) 

2~  — 2 

Ul+Ulti2+U2 


A  R2  R12 


{0,1,2}  {22,10,01,11, 
02,12,20,21} 


R2  A  R12 


{0,1,2}  {22,10,01,11, 
02,12,20,21} 


etc.,  (different  for  M  ,  M2> 


=  revised  model  M2  =  standard  model 
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The  important  thing  to  notice  from  Table  4.3  is  that,  for  policies 
where  repair  is  only  done  on  one  component  at  a  time  in  a  given 


state,  V  is  the  same  for  and  M2.  However,  for  policy 

AARj^*  which  lets  the  system  fail  and  repairs  both  components, 

aar12 

V  under  is  lowey  than  that  under  M2>  The  lower 

variance  on  the  component  lifetime  of  number  1,  now  with  an 
increasing  failure  rate,  makes  the  multiple  repair  policies  more 
attractive  relative  to  single  repair  ones  which  are  always  optimal 
for  exponential  cases.  Thus,  one  would  expect  that  it  is  no  longer 
true  that  single  repair  policies  are  always  optimal  and  indeed  it 
is  so: 


Let  =  K2  =  =  p2  =  10  p  *  2  in  Example  4.3. 


K  +p  K„+p 

Table  4.3  =*•  — —  =  1.2  =  - 

V1  V2 


1  +  1=2 


MR12  (20)  (22)  440 

-1  "  300+150  450 


so,  a  policy  with 
in  0  must 
be  optimal. 


This  concludes  the  results  for  the  Degradation  Model.  A 
sumnary  along  with  further  conclusions  and  comparisons  to  other  models 
is  found  in  Chapter  VII  (Conclusions). 
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CHAPTER  V 


THE  NON- INSTANTANEOUS  REPAIR  MODEL 


1.  Description  of  Model 

Up  to  now,  in  both  the  Basic  Model  and  the  Degradation  Model, 
attention  has  been  focused  on  the  components.  Such  factors  as  what 
states  they  can  be  observed  in,  what  type  of  system  they  make  up, 
their  lifetime  distributions  and  mean  lifetimes  have  been  looked  at. 
The  repair  assumption  throughout  has  been  that  it  is  done  instan¬ 
taneously  and  can  be  done  as  often  as  desired  (unlimited  service) . 

The  purpose  of  the  Non-instantaneous  Repair  Model  is  to  treat  cases 
where  repair  is  non-instantaneous  (exponential  service  in  most  cases) 
and  the  number  of  servers  may  be  finite.  Comparisons  to  the  Basic 
Model  (instantaneous  case)  can  then  be  made. 

Given  the  purpose  of  investigating  repair  assumption  effects, 
component  assumptions  were  chosen  to  be  as  simple  as  possible,  i.e., 
Basic  Model  assumptions.  The  Degradation  Model  could  be  modified 
to  incorporate  non-instantaneous  repair  in  a  similar  fashion  but  the 
number  of  states  would  be  large,  results  would  be  hard  to  come  by 
except  on  the  computer,  and  attention  would  be  diverted  from  the 
repair  aspects. 

The  Non-instantaneous  Repair  Model  has  the  following  Coherent 
System  Repair  Model  parameters: 

States:  each  component  can  be  in  one  of  three  states: 

working,  failed  and  under  repair,  or  failed 
and  not  under  repair. 


Repair:  non- instantaneous  (exponential  or  Erlang) ,  when 

completed  it  brings  a  component  to  "new"  condition. 

The  number  of  servers  is  1  .1  s  <*>  and  each  has 

identical  service  distributions,  R  ~  1  -  e 

s 

mean  a. 


Component  Lifetimes:  exponential,  ~  1  -  e 


Note  that 


gives  the  instantaneous  repair  (Basic 


Model)  case. 


Again,  given  the  increased  complexity  of  the  model,  results 
are  much  more  difficult  to  obtain.  As  with  the  Degradation  Model, 

V results  only  are  looked  at.  Section  2  considers  the  case  of 
identical  components  (k-of-n  system),  no  fixed  charge,  exponential 
repair  and  a  single  server.  In  Section  3,  results  of  Section  2  are 
compared  to  multiple  server  results.  The  final  section  demonstrates 
other  possible  formulations  of  the  model  to  include  Erlang  service, 
non-identical  servers,  or  component  degradation  states. 

Unlike  the  Degradation  Model's  k-fold  increase  in  the 
number  of  parameters  over  the  Basic  Model's,  the  Non- instantaneous 
Repair  Model  requires  only  the  addition  of  three  new  parameters  to 
completely  specify  the  model  (in  the  exponential  repair  case). 

These  are: 


(1)  mean  repair  time  a  0 

(2)  labor  cost  l | server] unit  time 

(3)  number  of  servers,  s  >  1 
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in  addition  to  the  Basic  Model  carryovers: 

(4)  fixed  charge  L  0 

(5)  penalty  cost  for  system  failure  p  >  0 

(6)  mean  lifetime  of  component  i,  p  =  i  =  1,  n 

1  Ki 

(7)  number  of  components,  n 

(8)  cost  to  repair  ith  component,  K^,  i  =  1,  n 

(9)  type  of  system  (specification  of  states  for  which 
penalty  is  incurred)  -  series  case  impossible  as  with 
Basic  Model. 

The  objective  is  to  minimize  V  ,  the  long  run  expected 
cost  per  unit  time.  It  is  interesting  to  note  that  if  =  0  =  1 
and  p  =  o,  the  objective  becomes  minimizing  the  fraction  of  time 
the  system  is  failed  as  is  used  by  Smith  I29J  in  his  Optimal  Repair 
of  a  Series  System  model. 

A  state  in  this  model  can  be  denoted  by  a  vector  s  of  2 
parts;  s  =  < s^,  S2 >  where 

s^  =  vector  of  which  components  are  working 

82  =  vector  of  which  are  in  service. 

| Sj |  £  s  =  number  of  servers,  y  Sj  C  Ui  ....  n).  Changes  of 
state  occur  when  either  one  of  the  working  components  fails  or  repair 
on  one  of  the  components  in  service  is  completed.  At  such  an  instant, 
decisions  can  be  made  to  repair  some  subset  of  the  components  which 
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are  failed  but  not  in  service  if  the  servers  are  not  all  busy. 


Once  repair  is  started  on  a  component,  it  must  be  completed.  In 
the  special  case  of  a  k-of-n  system  with  identical  components, 
discussed  in  Sections  2  and  3,  the  vector  s  has  only  two 
components,  s  =*  <  i,  j)  where  i  =  number  of  working  components; 
j  *  number  of  components  in  service,  i  +  j  £  n,  j  £  s. 

The  total  number  of  states  in  an  n-component  Non-instan- 
taneous  Repair  Model  is  at  most  3n  in  the  case  of  s  ^  n.  In 
cases  of  small  s,  the  number  can  often  be  considerably  lower  than 
that,  although  still  greater  than  the  number  of  states  in  the 
corresponding  Basic  Model.  Thus,  although  results  here  are  just 
as  difficult  (if  not  more  so)  to  come  by  as  for  the  Degradation 
Model,  on  the  computer  they  will  be  easier  due  to  the  lesser  state 
space  enlargement. 

Repair  times  being  exponential,  each  state  as  defined 
previously  has  exponential  holding  time  so  this  model  qualifies  as 
a  continuous  time  Markov  decision  chain  with  infinite  planning 
(time)  horizon.  Notation  from  the  Basic  Model  in  Figure  2.1  all 
applies  here  with  the  addition  of  repair  parameters  s,  l,  a 
defined  previously.  The  Markov  chain  specified  by  the  Non- 
instantaneous  Repair  Model  with  given  parameters  and  exponential 
repair  is  as  follows: 
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State  Space: 

states  -  which  components  are  working,  failed,  or  in  service 
s  -  <  s ^ ,  s2  >  where  s^,  U  f2  C  {1,  . . . ,  n} 
s^  =  which  components  are  working 
§2  *  which  components  are  in  service  (|§2|  Is) 

If  system  =  k-of-n  and  components  are  identical,  then 

s  *  <  i,  j  )  ,  i“  number  of  working  components 

j  -  number  in  service  j  £  s 

i  +  ]  <n  . 

Decision  Space: 

Let  n  =  {1,  ....  n}.  Possible  decisions  in  state 
?-<?!,  s2)  are 

s 

where 

(i)  ^  C  ~  u  ?2  (repair  failed  components 

not  in  service  already) 

and 

(ii)  |  ft  U  g2 1  £  s  (number  in  service 

£  number  of  servers)  . 

Restrictions: 

-  ft  j4  <p  if  system  down  in  s  and  no  components  under 

s  ~  - 

repair. 

-  ■  $  (Rn  -  A  *  "do  nothing")  if  §2  -  s. 

§ 
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Transition  Structure: 


The  transition  matrix,  Qq:  (assuming  6  :  in  s). 

§ 

Fix  s  -  ( slt  f2> 


^0^s,<  S.  ,8,Ufl  -1) 
~  ~  X  ~  l  S 


<Vs,t 


£  x  +(i/o)is,un 
j€s1  J  '2  § 

1/a 

r  •  1  6  si 

l  i  +a/o)\s  un 
jEs1  J  2  5 

Y  ‘  1  e  ?2  U  “s 

0,  if  t  *  <s1  ~ 

1,  s2  U  ),  some 

or  f  (  Sj,, 

!2  u  ns  ~  i>’ 

some  i  €;  s0  u  ft 
-2  s 

i 

%S 


A.+(l/a)|s2Unsj  • 


Coat  Structure: 


'L(l  -  ij  )  +  l  \  +  l»2  uflJ  *  *  *  Q1  s  * 
ns  iea  1  2  5 

-  S 


if  system  up  in  s 


0,8 


L  +  P  +  I  Ki  +  I3?  u  nJ  *  1  ‘  Q1  a 
iSfl  1  2  '  i,§ 


if  system  down  in  s  and  s  is 
a  min  cut  set  of  the  system 


where 


s 


t 

i 


i, 

o. 


P‘s  "  ») 


Oblective  Function: 


*  6 

Vg 

*  6 

V  1 


(any  possible  optimal  6  will  form 
a  Markov  chain  with  a  single 
irreducible  set  of  ergodic  states 


The  following  examples  are  in  the  cases  of  identical  components 
with  "s"  servers: 


Example  5.1 

:  n  =  2,  s  * 

2, 

parallel,  L  * 

0 

states: 

2,0  1,0 

1,1 

0,0 

0,1 

decisions: 

A  A,RX 

A 

R1,R2 

R1,A 

cost : 

0  0,K+£ - - 

rr 

0 

K+Ho+p, 

K+lto+p, 

A+o 

JL 

2K+{.0+p 

ta+p 

R^  =  initiate  repair  of 

i 

components . 

Notes: 

state  0,2  is  not  listed  because,  although  it  is 
theoretically  feasible,  it  will  never  be  reached  given 
a  starting  state  with  all  components  up  and  the  given 
decision  possibilities. 
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in  state  0,1,  even  though  the  system  is  failed,  the 
decision  to  "do  nothing"  is  allowed  since  repair  has 
already  been  initiated  on  some  other  component  and  there 
is  no  way  to  get  the  system  operative  instantaneously 
under  this  model.  In  state  0,0,  however,  some  form  of 
repair  is  required. 

The  penalty  cost  is  incurred  once  each  time  the  system 
fails.  This  is  due  to  the  fact  that,  once  the  parallel 
system  fails,  all  components  are  failed  so  the  next  state 
change  must  be  due  to  a  service  completion,  sending  the 
system  to  a  working  state. 


Example  5.2:  n 

states:  3,0 

decisions:  A 

cost:  0 


3,  s  *  1,  k  =  2(2  of  3  system),  L  =  0 


system  up 


system  down 


2,0  (2,1)  1,0 


1,1 


/since  \ 

A,R^  A  RJ  system  | 


\failed/ 


O.K+i*- 


2  X+o 


-1 


K+p+i- 


X+o 


-1 


p+H* 


X+o 


-1 


0,l' 

A 


ia 


(i,  j)  denotes  a  state  which  will  never  be  entered. 


Notes: 

unlike  the  Basic  Model  for  a  k-of-n  system  with  k  >  1, 
states  for  which  there  are  less  than  k  -  1  components 
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working  can  be  reached  due  to  the  non-instantaneous  repair 

The  penalty  cost  is  now  incurred  each  time  the  system 
enters  a  state  in  which  k  -  1  units  are  working.  This 
occurs  either  at  a  system  failure  or  at  the  instant  of  a 
service  completion  still  leaving  the  system  down. 

There  are  actually  only  two  possible  policies  here: 

a)  do  nothing  when  2  components  are  left,  or 

b)  repair  when  2  components  are  left. 

under  a):  #={2,0  1,0  0,1} 

2,0  1,0  0,1 

0  10 

-V-S-  0  -1— 

X+l/o  X+l/o 

0  10 

These  examples  will  appear  later  in  the  chapter. 

In  the  parallel  case,  the  instances  of  penalty  cost 
assessment  are  clear  -  whenever  the  system  fails.  For  the  k-of-n 
model,  the  fact  that  states  with  less  than  k  -  1  components 
working  can  be  reached  means  that  this  penalty  could  be  assessed 
several  times  while  the  system  is  still  down,  how  often  depending 
upon  the  decision  the  modeler  makes  on  which  states  to  apply  p 
in.  If  it  is  desired  to  actually  have  p  assessed  only  once  per 
system  failure,  this  can  be  accomplished  by  dividing  the  states 
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with  k  -  1  units  into  two  categories:  those  due  to  system 
failure  and  those  not.  I  assume  that  the  p  gets  incurred  in 
such  states  no  matter  what  the  cause  of  entry  to  stick  to  the 
model  framework  introduced  previously. 

2.  Exponential  Service,  Single  Server  Results 

In  this  section,  we  look  at  the  Non-Instantaneous  Repair  Model 
in  which  the  components  are  identical,  L  =  0,  and  service  times 
are  exponential  for  the  case  of  a  single  server  (s  =  1) .  A  policy 
is  found  which  is  V optimal  for  first  the  parallel  case  and 
then  the  k-of-n  case  for  k  <  n.  Changing  behavior  of  this  policy 
for  variable  k  and/or  n  is  looked  at. 

Some  notation  is  now  needed: 

Definition:  Let  the  policy  ^  (for  the  aforementioned 

l , 

i  single  server  model)  denote  the  policy  which  repairs  whenever  the 

j  number  of  working  components  is  _<  j  and  the  server  is  idle. 

f 

i  Let 


Let  f,  be  defined  so  that:  f 
k  1 


1. 


=  (k-1)  io^1 

k  uk  2(u+(k-l)o) 


k  >  2  . 


The  theorem  giving  the  general  V  ^  optimal  solution  can  now  be 
stated: 


Lemma  5.1:  Suppose  we  have  the  Non-instantaneous  Repair 
Model  with  a  single  server.  Assume 

(i)  identical  components,  k-of-n  system 

(ii)  L  =  0  (no  fixed  charge) 

(iii)  exponential  service  times. 

Then: 

(A)  the  V  i  optimal  policy  is  among  k-l<_j<_n-l, 

policy  being  defined  previously, 

(B)  <  R^"1^ 


11=111 


j+1 


fk(K 


+  2-a) 


z . 

_JL  _ 

J  j+i 


MZj-i 
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Proof :  Consider  the  k-of-n  model. 

states:  (n,0),  (n  -  1,0),  ....  (k,0)(k  -  1,0) . (1,0) 

decisions:  A  A,  R^,  ...»  A, 

and 

(n  -  2,1),  (n  -  3,  1),  ....  (0,1) 


do  nothing 


States  where  decisions  have  to  be  made  are  (n-1,0),  (n-2,0),  ....  (k,0), 
options  being  to  repair  or  to  do  nothing.  This  gives  2  possible 
policies.  However,  these  yield  only  n  -  k  +  1  different  ergodic 
chain  structures  in  the  underlying  Markov  chain,  thus,  only  n  -  k  +  1 
policies  as  far  as  V_^  optimality  is  concerned.  These  are  precisely 
the  ones  mentioned  in  part  A.  of  the  theorem: 


_ State _ 

Policy  (n-1,0)  (n-2,0)  ....  (k+2,0)  (k+1,0)  (k,0) 


R(k-1) 

R(k) 

R(k+!) 

r(A-2) 

R(n-!) 


■<- 


A 

R, 


A 

R, 


A 

R 

R 

R 

R 


1 

1 

1 

1 


To  compare  the  R^,  the  V_^'s  must  be  computed  for  each. 
Table  5.1  gives  such  quantities  for  varying  j  and  k. 
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If  k  is  fixed  and  j  >  k,  then 


j  j-1 

z  (K+ioWjO^r12  z  (K+ioHCj-l)!5-^ 
_ _k_  <  J~1 _ ]k_ 


(K  +  ta)  !i  _  -j-lVi  <  2s.J'1a--Ui  !±ti  _  112. 

(K  +  j+1  _  ffc  |_  j+1  j 


But ,  since 


the  above  is  true  ** 


(K  +  ia)uj  ti _ H—  z  1  < 

(K  +  ^  j+1  *^1  f  .(j+l) 


(.1-1)! 


U<yJ~  >  fR(K  +  ia;  Zj^  , 


our  desired  result  B. 


Collecting  all  the  non-j -dependent  terms  on  the  left  hand 


side  gives  R(j)  <  R^-15  «*■ 


151 


m 


(i) 


-BL 


J±L 


fk.(K+ia) 


!L--y_z 
J  J+1  J-1 


4  u(J> 


Two  useful  properties  of  u(j)  are  now  proved. 


Lemma  5.2:  Let 


u(J) 


.  J+1 _ 

oj_1(j-l)! 


u 

j+1 


z . 
3 


-1 


where  z^  has  been  previously  defined  both  recursively  and  in 
general  form.  Then 


(1)  u(j)  >0vj  >1 

(2)  u(j)  -  u(j-l)  >  0  v  j  >  1  . 


Proof :  u(j)  _>  0 


j+1  “j-1  - 


>  0. 


Z0  =  Z1 


V  +  cr, 


thus. 


V  +  o 


u. 

2 


> 


0 


so,  u(l)  _>  0  .  Now,  suppose  u(j-l)  0. 
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Notice  that  for  any  1 


thus. 


u(i)  >  0  ~ 


1 

1 


*  Vi  i  o 


+  CTZ, 


z,  >  0 


5i-l  “  i+1  zl-l  - 

"  JT  +  zi-i(°  -  m|i° 


It  is  now  sufficient  to  show 


'f+zJ-i(o-3«)i0 


to  prove  (1). 


j  +  zj-i  (° '  j«)*  i ^  (4t  +  °zi-i) +  °  (tt  +  "j-2) 


'1  1 

X 

yJ_1  +  z  /  y 

j  (j+l)(j-l) 

*r  0 

j-l  +  Zj-2  (a  '  J+l 

>  uj 

1 

x  fx 

y"*"1  +  .  (a  y  \ 

_  H 

(j+l)  (j-l) 

»  O 

j-l  j-2  j  / 

>  0 


>  0  by  induction 


Now,  let  A ( j )  >=  u(j)  -  u(j  -  1);  defined  for  j  2. 


A(j)  =  u(j)  -  u(j  -  1) 


J+l 

li  _  ji_  z 

j 

P.1-1  P 

oj_1(j-l)! 

j  j+1  j-1 
-  - 

cr3-2  (j-2) ! 

|  j-1  j  j-2 

1 

(j-DIo3'1 


J±i  z 
j  *J 


liZ 


j-1 


-  a(j  -  l)(j) 


j- 


_  W  2  I 

j  Zj-2( 


-nJ'1 


(j-l)!o 


(J  +  1)  -J-  -  pz 


j_i  -  o  •  j  •  Zjl  +  0(j-l)pZj_ 


4  *1  P3 

since  j  ”  y  +  azj_i>  J  1  1» 


-1  'Ha3'1 


(j-1)  !a 


(j  +  1)  j  +  “  h)  +  a(j  ~  1>  VIZj_2 


-1 ^  in3"-1 


(j-1) ! o 


Jj-  +  ap3_1  +  a2(j  -  1)  z j_2 


>  0 


Lemma  5.1  along  with  Lemma  5.2  allow  us  to  state  the 
following  optimal  policy  form: 
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Theorem  5.3:  Suppose  we  have  the  Non-instantaneous  Repair 
Model  with  a  single  server  and  the  other  hypotheses  of  Lemma  5.1. 


Let 


for  1 
k  -of- 


u(j) 


_ J+l 

oj-1(j-l)! 


9 


<_  j  <_  n  -  1  and  u(0)  ■  0,  u(n)  =  °°.  Suppose  the  system  is 
n.  Then  the  V  optimal  policy  is: 


,<j> 


if  u(j)  < 


_EiL 


f,  ♦ (K+io) 


<  u(j  +  1), 


i  >  k 


.  (k-1) , 


if  0  < 


-E!L 


fk(K+to) 


u(k) 


Proof : 


Know  from  Lemma  4.1  that 


R(j)  R0“D  «. 


- EU -  > 

fk* (K+io) 


_ J+l  - 

oj_1(j-l)! 


j-1 


u(j)  • 


Lemma  5.2  "*  0  <  u(l)  <  u(2)  <  •••  <  u(n  -  1)  which 


implies  the  given  optimal  solution. 


□ 


: 


Schematically,  given  a  k-of-n  system,  s  =  1,  identical 
components 


u(l)  •••  u(k)  u(k+l)  u(k+2)  •••  u(n-2)  u(n-l) 

- * - « - * - * - M - ► - EE 


• - K - 

0 

— M - * 

«  1 

1  1 

1  1 

i 

i 

‘  fk(K+ia) 

R(k-!) 

R(k)  ; 

\ 

Roc+i)j . . 

(n-2)  !  (n-i)  °ptf“al 

♦  •  R  ;  R  policy 

Thus,  given  a  fixed  k,  factors  which  favor  more/(less)  repair  are: 

(a)  (low) /high  penalty  cost,  p  for  system  failure 

(b)  (high) /low  component  repair  cost,  K,  and/or 

(high) /low  expected  labor  cost  per  server  per  repair 
job  completed,  la  . 

This  is  because  the  "u"  values  are  only  functions  of  p  and  a, 
not  of  system  and  repair  cost  parameters.  As  o  0  (close  to 
Instantaneous  repair),  the  u-values  all  approach  infinity  except 
u(l)  =  p.  This  means  that  for  small  enough  o,  one  will  never 
repair  until  there  are  only  k  left  in  a  k-of-n  system,  a  familiar 
Basic  Model  result. 


u(l) 


J5L 


k  =  1 (parallel) 


,(0). 


,(D 


(f 2*1) (K+fca) 

u(2)-u(3)-*» 
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In  the  parallel  model,  the  a  =  0  result  simplifies  precisely 
to  the  Basic  Model  result  since  the  optimal  policy  there  needed 
only  a  single  server.  For 


V 

c 


k  >  1 
o  «  0 


(k-1) ! 
k-1 


k-1 


which  must  be  incorporated  into  the  u's.  This  done,  lim  f,  •  u(j)  =  » 

o-K)  k 

for  j  ^  k  and  is  <  “  for  j  =  k;  therefore,  the  limiting  a  *  0 
policy  is  the  same  as  for  parallel.  The  next  example  illustrates: 


Example  5.2:  n  =  3,  s=l,  K=2,  L=0 
Possible  policies  are  and  i.e.,  if  in  a  state  with  two 

components  working  and  an  idle  server,  either  repair  or  don't  repair. 

optimal  optimal 

^ - ►  — - - EH - - 

(o/a+p) (K+ia) 

"  2  7 

^-+2ya+3  a 


u(2)  -  i 


optimal 


R<2> 


optimal 


lim  f.  •  u(2)  =  £ 
o->0 


as  a  -►  0 
(close  to 
instantaneous 
repair) 


R^  optimal 


0 


_ , _ EH _ 

(o/o+y ) (K+io) 


"takes  so  long  to  repair  that 
might  as  well  let  system  fail  first" 


u^)-^ 

V1 


as  o  -*■  00 

(large  repair 
times) 

-*■  unlikely  case 
in  practice. 
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In  the  remainder  of  this  section,  the  problem  of  how  the 
aforementioned  V_^  optimal  policy  is  affected  by  the  type  "k" 
of  a  k-of-n  system  is  treated.  Theorem  5.3  states  that,  in  a 
k-of-n  system,  the  V  optimal  policy  is: 

(2)  R(j),  if  u(j)  <  -^(ScW  <  u(j+1)> 

and 

(3)  E‘k-»,  it 


The  only  k-dependent  quantity  is  f^,  except  for  the  value  of 
j  at  which  (2)  is  cut  off  and  (3)  holds.  A  study  of  f^  behavior 
will  yield  any  system-related  policy  changes. 


Lemma  5.4:  Let  f  =  1  and 


f  -  (k-1) !o 

k  '  k-2 


k-1 


k  >  2 


U  <u+(k-l)cr) 


Let 


i+/i*4 


Then 


and 


fk  "  fk+l  <  +  )  for  k  <  k* 
fk  "  fk+l  (+)  f0r  k  "  k*  • 


158 


Proof :  f 


1  >  -  f2*  (k  >  1  always).  Suppose  k  ^  2. 


fk  <  fk+l  ~  (k  "  l)!ok~1vk-1(w  +  ko)  <  k!okwk_2(p  +  (k-l)o) 
**  u(g  +  ko)  <  ko(y  +  (k  -  l)o) 


**  g2  <  k(k  -  1)  o2 

~  k2  -  k  -  O^-)2  >  0 
o 

iJZZ 

*  a  r-1 

**  k  >  k  =  - ^ -  by  quadratic  formula.  □ 

Notes: 

(1)  k*  >  1  (f1  >  f2) 

u  * 

(2)  As  —  increases,  k  increases. 

Consider  a  partition  of  [0,  <*>)  by  u(j),  1  £  j  £  n  -  1: 


0  u(l)  u(2)  •  •  •  u(n-l) 

- - X - X - X - 


These  u's  do  not  vary  with  k  although  as  k  increases,  only 
u(k) . u(n  -  1)  are  meaningful  in  determining  optimal  policy. 


Optimal  behavior  will  be  determined  by  the  position  of 


PH 

fk(K+*o) 
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and,  thus,  will 


on  this  line. 


A  smaller 


raises 


_ EH _ 

fk(K+*o) 


cause  earlier  repair  (when  fewer  components  are  failed)  if  it  is 
changed  enough.  Similarly,  a  larger  f  means  a  wait  until  more 
components  have  failed  before  commencing  repair.  Lemma  5.5  states 
this  precisely. 


Lemma  5.5:  Suppose  we  have  a  Non-instantaneous  Repair  Model 

with  a  single  server  and  identical  components  and  the  other  assumptions 

of  Lemma  5.1.  Let  p,  a,  y,  K,  and  Z  be  given.  Let  represent 

★ 

the  model  for  a  k-of-n  system,  1  _<  k  _<  n.  k  is 

1^4 

_ a 

2 


Suppose  policy 

R(i) 

was 

optimal 

for 

V  1 

1  k. 

Then, 

if 

* 

k  <  k 

,  R(j) 

is 

optimal 

for 

Vi* 

some 

j  >  i 

if 

* 

k  >  k 

,  R^ 

is 

optimal 

for 

Vr 

some 

j  <  i 

Proof :  Obvious  from  Lemma  4.4  and  previous  remarks.  A  final 
example  illustrates: 


Example  5.3:  n  =  4,  p,  K,  i,  y,  a  fixed.  L  =  0,  k-of-n 


system  where  k  =  1,  2,  or  3. 


u (1 )  =  2o  +  U  =  5 


f  =  1  - EH - 

1  f^K+io) 


u(2)  =  13y 


_ £H _ 

f2(K+lia) 


16 


u(3) 


_ EH _ 

f  3  (K+£a) 


30 


R(0) 

u(l) 

R(1) 

1 

u(2)  R(2) 

1 

u(3) 

R(3> 

1 

k 

=  1 

0 

1 

10 

1 

20 

1 

30 

*  R(0) 

opt imal 

R(1) 

1 

u(2)  R(2) 

u  (3) 

r(3) 

1 

k 

=  2 

0 

1 

10 

20 

1 

30 

r(2> 

optimal 

r(2) 

1 

1 

u(3) 

r(3> 

X 

k 

=  3 

0 

1 

10 

1 

20 

30 

"  r(3) 

optimal 

In  this  example  of  a  system  for  which  the  mean  repair  time 
is  small  compared  to  the  mean  component  lifetimes,  as  k  increases 
the  number  of  failed  components  at  which  repair  is  started  is 
increased.  Note  that,  if  f^  had  been  increasing  in  k,  as  would 


have  been  true  for  a  small  y/o  ratio,  then  increasing  k  would 
have  meant  waiting  longer  before  initiating  repair  up  to  the  point 
where  the  system  fails  in  whicli  case  further  increase  of  k  forces 


TW*T 


a  lesser  wait  due  to  the  necessity  of  repair  upon  system  failure. 
So,  generally. 


p/o  large  =*  short  repair  times  repair  sooner  as  kt 

compared  to  components’ 
lifetimes 

p/o  small  =*  long  repair  times  =*  repair  less  as  k+  . 
compared  to  component's 
lifetimes 


Example  5.4:  p  =  1  0=1  p  =  6.1  K  =  1  t  -  1/2 


u(l)  =  3 


f  =i  - — -  =42 

rl  1  f,(K +la) 


u(2)  =  5~ 


JE1L 


2  2  f2(K+^a) 


8.4 


u(3)  =  8^ 


fn  = 


JE1L 


3  3  f3(K+Ho) 


6.3 


R(0) 


,Cl) 


,(2)  r,  (3) 


u(l)  “  u(2)  u(3)- 


10 


k  =  1 
optimal 


.(1)  r(2) 

\  u(2)  K  u (3)- 


,(3) 


10 


k  =  2 
(3) 

r'1  optimal 
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,(2> 


,(3) 


"~u  ( 3 )  - 

— U — 


10 


k  =  3 

R^  optimal 


3.  Exponential  Service,  Multiple  Servers 

In  Section  5.2,  a  single  server  version  of  the  Non-instan- 
taneous  repair  Model  for  an  identical  component,  no  fixed  charge, 
k-of-n  system  was  treated.  The  V_^  optimal  policy  was  found  to 
be  among  k-l£j<_n-l  where  signifies  a  decision 

to  repair  whenever  the  server  is  free  and  the  number  of  working 
components  is  j  or  less.  The  purpose  of  this  section  is  to 
investigate  what  happens  for  the  same  model  but  with  multiple 
servers.  For  the  instantaneous  repair  (Basic  Model)  case,  it  was 
never  optimal  to  repair  more  than  a  single  unit  at  a  time  in  an 
ergodic  state  so  at  least  if  one  was  only  interested  in  V  the 
optimal  policy  for  multiple  servers  would  be  the  same  as  if  there 
was  only  one.  The  bulk  of  this  section  is  spent  trying  to  see 
whether  that  is  still  the  case  for  non-instantaneous  repair  either 
in  general  or  in  some  cases. 

First,  consider  Example  5.1. 

Example  5.1:  n  =  2,  s  =  2,  parallel,  L  =  0 


states : 

2,0 

1,0 

1,1 

0,0 

0,1 

decisions : 

A 

A,R: 

A 

R1’R2 

RrA 

Possible  policies,  6,  giving  different  ergodic  structures  are: 
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1 


The  only  policies  which  require  only  a  single  server  are 
and  The  relationship  between  the  five  policies  is 

summarized  below: 


Figure  5.2  Policies  in  Example  5.1 


lower 


(better) 


higher 


V 


-1 


(worse) 


A  vertical  arrow  indicates  total  domination  by  the  policy  located 
higher  in  the  diagram.  Two  upward  "v"  arrows  indicate  that  the 
policy  below  is  dominated  by  one  of  the  policies  at  the  ends  of 
the  "v". 

In  this  case,  it  is  indeed  true  that  a  policy  which  requires 
a  single  server  only  is  always  optimal. 

Now,  suppose  we  have  the  same  model  with  two  servers  but 
now  with  .1  components: 
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Example  5.5:  n 
states:  3,0 
decisions:  A 


■  3,  s  =  2,  parallel,  L  * 

*  *  * 

2,0  2,1  1,0  1,1 

A,R^  A  A, A, R^ 


0 

*0,0  *0,1  0,2 
R^,R2  A,R^  A 


*  =  decision  to  be  made. 


There  are  now  quite  a  number  of  possible  policies,  but  only 
three  which  give  ergodic  structures  which  utilize  only  a  single 
server:  and  R^2^  as  defined  in  Section  5.2. 

Consider  the  feasible  policy  6  =  R^  -  R^  —  which  gives 
&  =  {3,0  2,0  2,1  1,1  0,2}.  Clearly  this  requires  two 

servers.  Computations  give 


V5  _  (w+cQ2 (K+fcq)+q2p 

-1  3  „  „  3 

^-+ap  +a  y+^ 

Also, 

Vr(2)  -  (u2+2au+2 a2) (K+la)+2q2p 

-1  3  ,2  n 

^-+ay^+2a  y+2a 

WR^  (y+a)  (K+£a)+op 
-1  '  2  2 

^ — l-  ya+a 

R(0)  K+lo+p 
y+a 
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Suppose  K  +  la  -  1;  p  ”  30,  o=l,  w  =  1 .  Then, 


31 

2 


15.5 


32 

2.5 


12.8 


12.19 


Notice  that  policy  6  is  better  than  any  of  the  possible 
single  server  options.  Thus,  for  this  case,  some  policy  (may  or 
may  not  be  6)  is  optimal  which  does  require  the  services  of  both 
servers.  Under  instantaneous  repair,  policy  6  would  not  have 
been  optimal  since  it  involves  repair  before  getting  down  to  one 
component  left.  Thus,  under  non-instantaneous  repair,  it  is  no 
longer  true  that  no  fixed  charge  =►  never  repair  more  than  one 
unit  at  a  time. 

The  section  is  concluded  with  an  obvious  result  but  worth 
nothing: 


Lemma  5.6:  Suppose  policy  6  is  V_^  optimal  for  a 

Non-instantaneous  Repair  Model  with  s-servers  (Ng)  but  it 

involves  the  use  of  at  most  only  s'  <  s  servers.  Then  policy  6 

is  optimal  for  the  Non-instantaneous  Repair  Model  with  only  s' 

servers  (N  , ) . 

s 


6 


Proof :  If  6  is  optimal  for  Ng,  then 


6  =  argmin  V'  , 
y€A(N  )  ~L 


A(N  )  being  the  policy  space  for  model  N  .  But  A(N  )  D  A(N  ,) 
s  s  s  —  s 

so  given  a  £  A  (Ngf), 


Va  >.  min  VY  >_  rain  Y  = 

y€A(N  ,)  "  YGA(N  )  -1 

s  s 


But 


6GA(N  ,)  so  min  VY  <  V6 

3  yga(ns,) 


so  =  min  VY 

y€A(Ngl) 


□ 


4.  Extensions  of  Model  to  Erlang  Service  Times,  etc. 

In  Sections  1,  2,  and  3,  exponential  service  times  were 
assumed.  As  in  Section  IV. 4,  with  component  lifetimes  in  the  Basic 
Model,  service  times  also  can  be  extended  to  Erlang  distributions 
in  the  Non-instantaneous  Repair  Model,  while  still  retaining  the 
Markov  decision  chain  structure.  For  many  applications,  an  Erlang 
repair  time  is  more  realistic.  Using  it,  one  assumes  the  longer 
time  the  repair  of  a  unit  has  been  going  on,  the  greater  the 
probability  of  its  completion.  (IFR  property). 
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Suppose  now  that  service  times  are  distributed  G  (t) 

— »® 
mo 

an  Erlang  distribution  with  shape  parameter  m  and  mean  o,  all 

servers  being  identical.  Then  if  R  is  the  service  time, 

R  =  R, + • • •  +  R  where  R  ~  exp  (l/o).  As  is  standard  procedure 

in  queuing  theory,  think  of  the  service  as  done  in  m  consecutive 

parts  with  completion  after  the  m1"^  stage  is  finished.  For  the 

general  Non- instantaneous  Repair  Model  with  non-identical  components, 

a  state  s  =  <  s, ,  . . . ,  s  )  where  s .  is  the  state  of  the  i1"*1 
1’  n'  i 

component  is  generalized  so  that  a  component  can  be  in  one  of 
m  +  2  states: 

1.  working 

2.  failed 

s  t 

3.  failed,  in  1  service  stage 
■ 

nri-2.  failed,  in  m^  service  stage. 

Decisions  only  need  be  made  concerning  a  component  if  it  is  failed, 
not  in  service  and  there  is  a  free  server.  The  usual  state 
simplifications  occur  if  components  are  identical.  If,  in  addition, 
we  assume  that  a  component  can  also  be  in  i  states  of  degradation, 
a  Degradation/Non- instantaneous  Repair  Model  is  obtained.  The 
total  possible  number  of  states,  (Jt  +  1  +  m)n,  is  t.ow  rather  large  and 
unworkable,  even  comoutationallv,  for  all  but  very  small  values  of  m,t,n. 
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Example  5.6  Degradation/Non-instantaneous  Repair  Model 
with  Erlang  Repairs 


n  -  2 

1  =  2 

s  =  1 


components  ~  exp  (A^)  Lj  ~ 

degradation  states  ~  exp  (A^)  ~ 

single  server,  repair  times  R  ~  G  (t) 

_L  3 

(so,  3  repair  stages)  JO 


exp  (A2) 
exp  (A*) 


states  s 


(s^  s2>  where  s^  s2  £  (-3,  -2,  -1,  0,  1,  2} 


-3  =  first  stage  repair 

-2  =  second  stage  repair 

-1  =  third  stage  repair  (last) 

0  =  new  (repair  completed) 

1  =  first  degradation  state 

2  =  second  degradation  state  (failed)  . 


Note  a  new  twist  to  this  combined  model  -  repair  could  be 
undertaken  on  a  working  component  in  a  lower  state  of  degradation 
and  the  component  could  become  further  degraded  or  fail  before 
service  is  completed.  All  kinds  of  decisions  on  what  to  do  in 
such  cases  can  be  treated  under  this  formulation. 
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CHAPTER  VI 


COMPUTATIONAL  METHODS  OF  PRODUCING  OPTIMAL  POLICIES 

1.  Introduction 

The  past  four  chapters  have  treated  four  distinct  types  of 
coherent  system  repair  models,  coming  up  with  theorems  which  either 
produce  a  general  optimal  policy  given  cost,  system  and  lifetime 
parameters  in  simpler  cases,  or  limit  the  number  of  possible  optimal 
policies.  Such  general  optimal  policies,  where  possible  to  obtain, 
are  clearly  the  most  desirable  results.  However,  for  many  cases 
the  theorems  developed  in  Chapters  II  -  V  do  not  apply.  A  partial 
listing  of  such  cases  would  include: 

(1)  Non  k-of-n  systems  with  or  without  identical  components. 
For  this  case,  in  the  Basic  Model,  the  V  ^  optimal 
policy  can  be  obtained  using  Theorem  2.4  if  there  is  no 
fixed  charge.  Otherwise,  nothing  applies. 

(2)  The  case  of  a  fixed  charge  L  >  0. 

If  components  are  identical  or  system  is  k-of-n,  then 
results  can  be  obtained  for  the  Basic  Model. 

(3)  Multi-server  cases  of  non-instantaneous  repair. 

(4)  Cases  where  there  are  many  degradation  states,  or  one 
has  Erlang  service,  component  lifetimes. 

Indeed,  while  Chapters  II  -  V  cover  many  cases  of  interest, 
there  are  many  more  possible  cases  not  solved  for  which  a  user  of 
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a  coherent  system  repair  model  would  want  the  optimal  solution. 

In  fact,  given  some  possible  application,  chances  are  it  would 
not  be  covered  by  any  of  the  theorems.  The  aim  of  this  chapter 
is  to  discuss  methods  of  solving  on  the  computer  a  given  coherent 
system  repair  model  which  can  be  formulated  as  a  Markov  ’ecision 
chain. 

The  next  section  discusses  two  general  types  of  algorithms 
which  could  be  used  in  optimizing  such  processes  by  computational 
methods:  policy  improvement  and  linear  programming. 

Section  3  describes  a  method  of  computing  optimal  policies 
for  the  Markov  coherent  system  repair  models  (or  any  other  Markov 
decision  chain  with  suitable  state  or  decision  space)  using  MINOS, 
a  non-linear  programming  code  developed  at  Stanford. 

Some  test  results  using  the  MINOS  solution  technique  will 
be  presented  in  Section  A. 

2 .  Possible  Algorithms 

There  are  two  possible  algorithms  which  can  be  used  in 
computing  a  gain  (V  optimal  policy  for  a  continuous  time 
Markov  decision  chain  (or  Markov  renewal  program)  with  infinite 
horizon  and  no  discounting.  These  are  policy  iteration  and  linear 
programming. 

Suppose  we  are  given  a  general  Markov  decision  chain  with 

(i)  states:  1,  2,  ...,  N  (indexed  i) 

(ii)  decisions :  D  =  set  of  possible  decisions  in  state  i 

(indexed  k) ,  finite 
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The  preceding  problem  can  be  reexpressed  as  a  linear  program 
K  k 

in  terms  of  variables  x^,  where  x^  =  probability  {state  =  i  and 
decision  is  k}.  The  optimal  x^  values  then  give  the  optimal 
randomized  policy  for  the  given  Markov  decision  problem,  i.e.,  values 
of  =  precision  k| state  i} 


The  LP  is  as  follows: 

c  c  k  k 

minimize:  l  l  x  r 

i  k 

subject  to:  [  -  l  l  p^  =0,  i  =  1,  2,  ....  N 

k  j  k  ] 

and  l  l  xi  vi  =  1 
ik 

xj  >  0  v  i,  k  . 
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The  objective  function  is  just  the  expression  for  V  of 

k 

a  given  policy  6  expressed  in  terras  of  x^  variables  while  the 
equality  constraints  are  a  rewrite  of  the  stationary  probability 
equations  for  a  policy  6: 


x  (I  -  p6)  =  Q 


)x .  = 

L  l 


Wagner  [34]  observed  (by  complementary  slackness)  that  a 
nonrandotnized  policy  is  optimal  over  the  class  of  stationary 
randomized  policies  in  the  case  where  each  optimal  policy  has  a 
single  ergodic  chain,  (as  is  true  for  coherent  system  repair  models). 
Thus,  in  an  optimal  solution  to  (P) ,  x^  >  0  for  only  one 
k  V  i  =  1,  . N.  This  indicates  an  optimal  decision  of  k  given 
state  i ( 6 ( i )  =  k) . 

k 

Not  all  states  i  need  have  x.  >  0  for  some  k.  Those 

l 

that  do  not  are  transient  states  in  a  V optimal  policy  and  those 

for  which  it  is,  are  ergodic.  Thus,  =  {set  of  ergodic  states  in 

optimal  policy}  =  {i  :  £  x^  >  0}. 

k  1 

The  second  method  for  finding  a  optimal  policy  is  policy 

iteration.  This  algorithm  has  the  advantage  that  it  can  handle  a 
countable  state  space  (N  =  °°)  while  N  <  <*>  is  needed  for  the  LP 
to  be  defined.  Also,  it  will  often  find  the  optimal  solution  in 


fewer  iterations  than  the  LP  will. 
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Policy  iteration  involves  repeating  two  steps,  value 


determination  and  policy  improvement  until  a  policy  6  with 
minimum  V  is  arrived  at.  Given  the  general  Markov  decision 
chain  and  the  fact  that  every  optimal  policy  defines  a  single 
ergodic  chain,  (in  a  coherent  system  repair  model),  the  two  steps 
are  executed  as  follows.  Suppose  one  starts  with  policy  6  . 

Let  u  =  <u^,  UN>  and  g  be  a  scalar. 

(1)  value  determination  -  solve  the  system  (given  6) 


r6(i) 


N 


-  I 

3-1 


<$(1) 

Pij 


u . 
J 


.  6(1) 

u  +  v . 

l  l 


1, 


and  u  .  =0  for  some  i  £  i  £  N  for  g,  u.  Since 

*  1o  0 

* 

P  R 

gs  ■  A  ■  ‘>nd  «  -  vv  i  •  °  • 

-1  p*Ql  0  -1 

this  g  =  g  ,  the  gain  value  for  policy  6. 


(2)  policy  improvement  -  using  the  current  values  of 


6  6 

g  ,  u  ,  find  another  policy  y  such  that  y^  =  what  y  does  in  i 


minimizes 

kGA(i) 


ii  i/f  k  N 


-  U.  }  Vi  =  1,  • .  • ,  N 
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If  y  :  5 ,  then  terminate;  6  is  optimal.  Otherwise,  go  back  to 
step  (1)  and  replace  6  by  y.  Continue  the  procedure  until 
x,  g  are  found  satisfying  the  termination  conditions: 


r6(1)  + 

1  j-1 


6(i)  .  6 (i) 

p..  u .  =  U .  +  V . 

1J  J  1  1 


i  =  1,  ....  N 


(t2) 


k  r  k  k 

r.  +  >  p..u.  >u.  +  v .  *  g 

1  FU  J  -  i  l 


V (i,  k)  pairs 


P 

Condition  (t^)  implies  g  =  V  while  (t2>  implies  6  is 
6  * 

V_^  optimal  (V  ^  =  g  ),  i.e.,  there  are  no  possible  "improvements". 
Given  a  finite  number  of  states,  termination  will  occur  in  a  finite 

number  of  iterations.  In  effect,  the  policy  iteration  method  is 

£ 

computing  V_^  for  various  6,  but  is  choosing  the  successive  6's 
in  an  efficient  manner  so  as  to  go  through  as  few  of  them  as  possible 
before  reaching  an  optimal  one. 

It  should  be  noted  that  the  linear  program  (P)  is  not  using 
the  same  algorithm  for  solution  as  the  policy  improvement  method 
mentioned.  However,  the  two  problems  are  intimately  related  in  that 
if  the  policy  Improvement  termination  conditions  and  a  desire  to 
maximize  g  are  put  in  linear  program  form,  the  LP  so  formed  is 
the  dual  problem  to  LP  (P).  Although  problem  (P)  is  the  more 
intuitive  formulation,  its  dual  (D)  turns  out  to  be  more  efficient 
computationally,  especially  in  cases  where  information  about  higher 
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levels  of  optimality  than  V  is  desired.  The  dual  problem  (D) 
is  shown  below  in  LP  form  with  g,  u  =  (  u^  ,  ....  >  as  variables: 


(D) 


minimize : 


(-  g) 


subject  to:  u. 


N 

l 

j=l 


k  k 

p  .  .  u .  +  v  . 

U  3  i 


g  <  ri  v  (i » k) 


where  g,  u^'s  are  unrestricted  in  sign. 

Two  well  known  results  from  duality  theory  will  help  relate 
the  problems  (P)  and  (D)  as  well  as  assist  in  developing  a 
computational  method  in  Section  3.  These  are: 


Theorem  6.1:  (Duality  Theorem)  [  7  ]  or  [34] 

(a)  If  both  the  primal  (P)  and  dual  (D)  problems 

possess  feasible  solutions,  the  primal  problem  has  an  optimal 

A 

solution  Xj,  j  =  1,  2,  ...,  N,  k  G  A(j),  the  dual  problem  has  an 

optimal  solution  g,  u.,  i  =  1,  ...,  N  and  )  )  r.  x.  =  -  g. 

i  “  “  l  i 

l  k 

(b)  If  either  has  a  feasible  solution  with  finite  optimal 
objective  value,  then  the  other  one  has  a  feasible  solution  with 

the  same  optimal  objective  function  value.  [So  the  optimal  objective  is 
(P)  or  (D)  is  V  in  a  coherent  system  repair  model  by  (b) 
since  clearly  (P)  has  a  feasible  solution  with  finite  objective]. 


Proof :  See  [7],  [34]. 

A  corollary  to  this  which  is  useful  in  relating  optimal  values 
of  primal  and  dual  variables  is: 
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Corollary  6.2:  (Complementary  Slackness).  Let  (P),  (D) 
be  expressed  in  the  matrix  form: 


(P)  maximize:  cx 


subject 

to  : 

Ax  < 

b 

x  •> 

0 

where 

1  c  1 

s  n. 

1  y ! 

=  m 

|x| 

=  n 

|b| 

=  m  . 

Let  x  ,  y  be  corresponding  feasible 

*  /  * 

Then  both  are  optimal  **  y  •  /  Ax  -  b 

*  /  *  t  \ 

and  x  #(y  A  -  c  )  =  0  . 


(D)  minimize:  by 

subject  to:  ytA  >_  c 
y  0 

|  A  j  =  m  x  n 

solutions  to  (P)  and  (D). 
i  =  0,  (dot  product  of  two  vectors) 


Proof :  See  [30]  or  [31]. 

Corollary  6.2  implies  that  whenever  a  constraint  in  one  of  the 
problems  holds  with  strict  inequality,  so  that  there  is  slack  in 
the  constraint,  the  corresponding  variable  in  the  other  problem  is 
zero.  This  result  will  allow  us,  in  the  next  section,  to  know 

k 

which  of  the  x^  variables  from  (P)  are  positive  from  the 
solution  of  the  dual  problem. 

Once  a  V  ^  optimal  policy  has  been  determined  for  a  given 
Harkov  decision  chain  using  either  policy  iteration  or  linear 
programming,  a  bias  optimal  policy  can  be  obtained  by  solving  an 
altered  Markov  decision  problem  again  by  either  policy  improvement 
or  linear  programming.  Given  the  original  Markov  decision  problem 
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and  its  associated  LP  (D) ,  the  altered  problem  is  defined  as 
follows: 


(i) 

(ii) 


(iii) 

(iv) 


states:  the  same  (1,  ....  N) 

decisions :  restricted  to  A',  where  A'  =  set  of  policies 
< i,6(i) >  such  that 


N 


“i-  Ki(1)u,  . » 

j=l  J  J 

where  u^,  ....  and  g  are  the  optimal  values  found 

solving  the  original  problem.  Clearly  the  optimal  policy, 
* 


6  from  the  V  step  is  in  A' . 

transition  probabilities :  p^. ,  the  same 
lc  — k 

cost  structure:  r^  is  replaced  by  R^,  where 


R*  =  (-R1)J  +  ((Q2)J.  •  1)  8*  -  (Q^J.  •  u  , 

U  =  <  Uj^,  . . .  ,  uN> 


being  the  solution  to 


•k  k  k 

(S  6  6* 

(I  -  Qj  )  u  =  rJ  -  qJ  g 

and 

UN  =  0 


* 

found  for  6 
in  V  ^  problem  . 


(v)  objective:  minimum  V  =  g  (which  gives  minimum  Vq 
in  original  problem) . 
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See  Denardo  [10] for  verification  of  the  fact  that  if  u  is 


chosen  as  shown,  that  the  V  ^  optimal  solution  to  the  altered 
Markov  decision  problem  indeed  gives  a  optimal  policy  for  the 

original  problem  in  the  case  where  every  V  ^  optimal  policy  defines 
an  irreducible  Markov  chain.  (All  coherent  system  repair  models  have 
this  property  -  see  Section  II. 2). 

For  a  general  problem  (arbitrary  Markov  decision  chain)  in 
which  some  V  optimal  policies  might  generate  multiple  ergodic 
chains  of  states,  the  aforementioned  altered  problem  may  not  yield  a 
Vq  optimal  solution  (a  case  never  encountered  in  a  coherent  system 
repair  model).  In  such  cases,  a  Vq  optimal  solution  can  still  be 
gotten  by  adding  a  third  step  which  takes  into  account  states  which 
are  transient  under  every  possible  policy. 

The  above  2  or  3  steps  can  be  continually  reapplied,  modifying 
r^s  and  restricting  decisions  appropriately,  to  get  V^,  V^,  ...  etc., 
optimal  policies  until  a  unique  policy,  "optimal",  is  reached.  For 
reasons  stated  in  Section  1.4,  optimality  is  considered  "optimal" 

in  coherent  system  repair  models  so  two  is  the  maximum  number  of 
V_^  problems  needed  to  be  solved  to  determine  an  optimal  decision 
for  every  state.  In  many  cases,  solving  one  problem  will  suffice 
in  getting  Vq  optimality  as  will  be  seen  in  the  following  two 
sections. 
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3.  Computing  Optimal  Policies  Using  MINOS  -  A  Linear  Programming 
Method. 

In  this  section,  a  linear  programming  method  for  solving  a 
coherent  system  repair  model  on  the  computer  using  MINOS  is  described. 
Although  policy  improvement  algorithms  may  on  the  whole  be  more 
efficient  and  require  fewer  iterations  to  reach  optimality,  there 
has  been  little  development  of  computer  codes  which  might  efficiently 
carry  the  method  out.  On  the  other  hand,  there  has  been  a  lot  of 
work  done  by  the  Systems  Optimization  Laboratory  (SOL)  at  Stanford 
on  linear  and  nonlinear  programming  codes.  See  [23]  and  references 
listed  there.  It  is  for  this  reason  that  an  LP  algorithm  is  used. 

MINOS  is  actually  a  code  developed  by  Michael  Saunders  and 
Bruce  Murtagh  [23]  for  solving  large  scale  nonlinear  programming 
systems  which  have  linear  constraints.  Of  course,  it  works  on 
linear  programs  as  well.  There  are  other  codes  which  are  designed 
for  linear  programs  and  can  handle  larger  sized  problems  (e.g., 

MPS  III  [referenced  in  [23]])  but  for  the  size  problems  tested  here, 
MINOS  is  sufficient.  Given  an  LP: 

minimize:  cx 
(P)  subject  to:  Ax  =  b 

x  _>  0  , 

MINOS  is  most  efficient  if  A  is  sparse  (has  lots  of  zeros)  and 
the  number  of  rows  isn't  too  large  (<  1500).  Coherent  system 
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repair  models  fit  these  conditions  for  small  numbers  of  components 
(under  10)  or  states  of  degradation.  In  general,  they  define  LP's 
which  have  many  more  columns  than  rows,  which  means  MINOS  can  handle 
larger  problems  than  might  otherwise  be  expected  in  solving  (P).  Of 
course,  in  a  general  nonidentical  component  model,  even  the  number 
of  rows  (states)  get  large  very  quickly,  e.g.,  if  there  are  l 
degradation  states  and  n-components ,  the  number  of  states  a:  (£  +  l)n 
The  number  of  columns  is  even  larger  by  a  factor  of  2n  since  in 
each  state  s,  one  can  in  general  decide  to  repair  any  subset  of 
the  failed  components  (2n  ^  possibilities),  and  there  is  one 
variable  per  possible  decision  per  state.  For  larger  problems, 
the  LP  code  MPS  III  could  be  used  but  clearly  for  any  moderately 
large  number  of  components  (even  _>  15),  model  simplifications  to 
restrict  the  number  of  states  and/or  decisions  must  be  undertaken 
before  the  problem  can  become  of  reasonable  size.  Such  theorems 
as  presented  in  Chapters  II  -  V  could  provide  such  assistance  in 
cases  of  identical  components,  k-of-n  systems,  or  certain  specific 
model  cases. 

Certain  of  the  models  can  be  solved  with  less  difficulty  than 
others  (due  to  smaller  number  of  states) .  The  Basic  Model  is 
easiest  with  the  Noninstantaneous  Repair  Model  requiring  only  a 
small  increase  in  state  space.  In  contrast,  a  Degradation  Model 
requires  a  dramatic  increase  in  states  and  for  even  small  numbers 
of  degradation  states,  would  be  too  large  to  do  even  on  the  computer. 
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r  — 


One  good  way  of  simplifying  the  problem  computationally 
would  be  to  break  the  system  down  into  independent  subsystems  of 
components,  the  subsystems  being  in  series,  i.e.,  s  =  {1,  ...,  n}  = 

B^  U  •••  UB^,  r  <  n  where  B^  O  B^  =  <f> ,  i  ^  j  and  the  system 
works  if  and  only  if  the  subsystem  corresponding  to  B^  is  working 
for  every  i.  The  idea  is  then  to  run  the  model  on  the  smaller 
subsystems  and  then  add  the  costs  from  each  one  to  get  the  overall 
result.  For  the  Basic  Model  and  V  this  can  be  done  as  a 

consequence  of  Theorem  2.1.  For  other  models  or  V^,  the  truth  or 
falsehood  of  this  result  is  an  open  question,  although  for  V  at 
least,  I  think  it  will  be  true. 

Before  stating  and  proving  the  aforementioned  theorem,  a 
clarification  is  needed  on  what  is  meant  by  independent  subsystems. 

In  the  Basic  Model,  the  states  of  the  system  depend  on  the  status  of 
each  component  as  well  as  the  status  of  the  system. 

Definition:  An  independent  coherent  subsystem  B_^  is  a 
subset  of  components  {1,  ...,  r},  r  £  n  which  form  a  coherent  system 
such  that  the  status  of  B^  is  unaffected  by  components  r  +  1,  . . . ,  n. 

Theorem  6.3:  Suppose  we  have  a  coherent  system  ff  of  n 
components  which  can  be  broken  down  into  p  independent  coherent 
subsystems,  B^,  which  are  in  series.  Then,  given  the  Basic  Model, 
the  optimal  long  run  expected  cost  for  the  system  is  the  sum  of 


183 


optimal  long  run  expected  costs  for  the  subsystems,  i.e., 

P 

V  ,  =  \  V  R  .  The  policy  is  the  same. 

j=l  j 

Proof :  To  follow  example  and  discussion. 

The  advantage  of  independent  series  subsystems  is  clear. 

Suppose  n  =  20,  i.e.,  there  are  twenty  components.  The  total 

20 

number  of  states  in  the  Basic  Model  could  be  up  to  2  =  1,048,576 

or  over  a  million  -  clearly  impossible  by  any  reasonable  computational 
standards.  Just  being  able  to  break  things  down  into  two  independent 
10-component  subsystems  would  reduce  the  total  states  needed  to 
2  x  2^  or  2,024  -  a  feasible  number.  Further  reduction  to  four 
5-component  subsystems  lowers  the  total  states  to  4  x  2  5  or  128. 

An  example  where  this  proves  useful  is  now  given. 

Example  6.1:  Suppose  a  system  consists  of  n  independent 

different  types  of  components  which  are  in  series,  i.e.,  the  system 

works  if  and  only  if  each  component  is  functioning.  One  possible 

way  to  improve  the  reliability  or  performance  of  the  system  is  to 

add  duplicates  for  each  component  type.  The  system  is  now  composed 

of  n  parallel  independent  subsystems  in  series.  If  the  level  of 

redundancy  of  component  i  is  denoted  by  r^,  then  the  total  possible 

n 

number  of  system  states  would  be  f]  (r.  +1),  r  >  1.  However,  by 

i=l  1  i 

modeling  each  subsystem  separately,  and  adding  V  ^'s,  one  gets 
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states 


away  with  running  n  subsystems  with  r^ ,  r^,  . ..,  r^ 
respectively,  almost  a  trivial  problem.  This  type  of  subsystem 
is  done  in  Markov  reliability  modeling  of  fault  tolerant  systems 
in  [24].  The  system  reliabilities  computed  for  each  subsystem  are 
then  multiplied  together  to  get  the  total  system  reliability. 

Fault  tolerant  systems  are  discussed  further  in  Section  7.4 
(Applications) . 

Proof :  (Theorem  6.3)  Let  the  independent  subsystems  of 

components  1,  ...,  n  be  reordered  so  that  B^  =  (1,  ...,  i^}, 

B„  =  {i.  +  1,  i,  +  i0),  ...»  B  =  {i  +  •••  +  i  +1,  ..., 

t  i  i  pi  p-1 

i ,  +  •••  +  i  }  .  Theorem  2.1  tells  how  to  find  the  V  ,  optimal 

1  p  -1 

policy  in  the  Basic  Model  case  for  a  coherent  system  by  testing 
all  subsets  of  components  for  which  the  system  operates.  The 
V  ^  optimal  policy  is  the  one  which  keeps  such  a  set  of  components 
operating  by  fixing  one  as  soon  as  it  fails  at  minimum  cost.  Let 
s  be  any  such  set  and  define  V  ^  g  as  the  V  ^  for  the  policy 
as  mentioned  above  which  keeps  only  the  set  s  of  components 
working.  Let  s  also  represent  the  policy  just  mentioned. 

Suppose  Sq  is  optimal  for  the  whole  system,  ,r/>.  It  is  desired  to 
show 


P 

y  v 

j=!  “1-s0rBlj 
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By  Theorem  2.1, 


V 

K.+P 

x  1  + 

y 

~1,s0  " 

^  u . 

i€s0:  1 

J. 

lso 

v1  ls 

down  in  IP 

V1  is 

up  in  ,'P 

If  stale 

so~  1 

is  U£ 

for  ,P,  then 

(sQ  '  i)  n  B. 

is  up  for 

subsystem 

V  j 

-  1,  • 

. . ,  p  since  the 

subsystems  are 

in  series 

If  state 

sQ  -  i 

is  down  for  PP>  then 

(sQ  -v  i)  O  Bj 

is  down 

for  B. 

3 

for  at 

least  one  j  since  the 

B .  1  s  are  in 

3 

series.  So 

has  to  be 

up  in 

!P  for 

the  policy  "keep  sQ  working" 

to  be 

defined. 

Thus, 

so  n  BJ 

is  up  in  B.  V 
3 

3  =  1,  • • • >  P. 

Thus, 

(s0  -  i) 

OB.  is  down 

3 

for  exactly  one 

B^ ,  the  one  which  i  is 

Thus 

P 

l 

j=l 


iGs„  HB . 

0  3 

(sonBj)'i 

is  down  in  B . 
-  3 


K.+p 

l 

Ui 


P 

+  l 

j  =  l 


.  ^  u . 

x&qGB.  1 

B.Os„  'i  is 
3  0 

up  in  B . 


I  v_i  _  nR  by  Theorem  2.1.  □ 

j  =  l  i,sQ  j 
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A  simple  example  is  now  solved  by  hand  using  the  LP  method 
following  which  the  procedure  for  solving  a  general  problem  using 
MINOS  is  presented. 


Example  6.2:  n  =  2,  parallel  system,  different  components 
L  =  0,  Basic  Model. 

Markov  decision  problem: 


States : 


12 


Decisions :  A  A.R^  A,R^  R^^,  R^ 


Transitions:  P 


12- 


Xl+X2 


Xl+X2 


R2  R1  R12 
=  P  =P  =P 
1-  2-  *0* 


A  Ro 

P^  =  P  1  =  P  2 

2*  0*  0* 


12 


.R”  _  ,R’  _  _J__  .  vA  =  vRl  =  -i-  •  /  =  vR2  =  -i- 

12  1  2  xi+x2  ’  1  0  X1  2  0  ^2 


=  =  V,2  =  V  1 


Costs : 


A  _  A  A  n 
r12  rl  "  r2  ° 


R2  R1 
rl  =  K2  5  r2  =  K1  ; 


R1  R2 
r0  =  K1  +  P;  r0  =  K2  +  P 


12 


\  +  k2  +  p 


Objective:  minimize  V  ,  then 
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Linear  Programming  Formulation  for  (p),  primal  problem. 

k 

8  variables  =  probability  {state  =  i  and  decision  is  k} 


,  0  0  2  0  1  1  2  12, 

(xl2,  xr  xr  x2,  x2,  XQ,  XQ,  XQ  )  . 


5  constraints  plus  x^  _>  0  v  i,  k  plus  objective  function. 


Then  (P)  can  be  written  as 

minimize:  cx 

subject  to:  Ax  =  b 
x  >  0 
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) 


<0 


9>  (objective  row) 


row 

S12 


1_ 

3 

2 

3 


0 

0 


0 

0 


1 

3 

2 

3 


51 

52 


0-10-1 


0  0 


0  1 


SO 


2 

3 


2 


2 

3 


1 


2 


2 

3 


NORM 


The  dual  problem  (D) 

can  be  written  as  follows: 

maximize:  g 

subject  to:  utA  c 


where  u  =  (  u-^j  uq)  and  g  are  unrestricted  in  sign. 


*  t 

We  know  max  g  =  min  cx  =  V  (duality  theorem)  and  x  •  (u  A-c )  =  0 

t  k*  r  * 

(complementary  slackness)  so,  u  A. ^)  <  c  ^  =  0  (i  £  <f  under  6  ), 

(A  , .  is  the  i.k*'*1  column  of  A). 

*  \  ^ 

Example  6.2  will  be  solved  as  part  of  Section  4. 
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F/G  15/5 


The  fundamental  principle  behind  MINOS  is  an  efficient  and 
reliable  implementation  of  the  revised  simplex  method  for  linear 
programming  (see  [  7  ]).  This  combines  established  sparse-matrix 
technology  with  stable  numerical  methods  for  computing  and 
modifying  a  triangular  factorization  of  the  usual  square  basis 
matrix  B  [see  references  in  [23],  p.  8]. 

For  usage  in  solving  purely  linear  problems,  the  following 
two  items  must  be  supplied  as  input: 

(1)  the  SPECS  file  -  to  specify  certain  run  time 
parameters 

(2)  the  MPS  file  -  to  specify  the  objective,  constraints, 
and  bounds  on  variables  in  standard  MPS  format. 

MPS  format  is  defined  under  the  title  "CONVERT  DATA"  in  IBM 
document  number  SH20-0968-1,  "Mathematical  Programming  System- 
Extended  (MPSX) ,  and  Generalized  Upper  Bounding  (GUB)",  pp.  199-209. 
The  following  SPECS  are  used  in  all  test  problems  in  Section  4: 


SPECS  FILE 


BEGIN  SPECS 

MINIMIZE  1  ... 

OBJECTIVE 

OBJ  J 

objective 

RHS 

RHS 

defines  RHS  vector 

BOUNDS 

BND 

defines  bounds  vector 

ROWS 

100 

max  number  of  rows 

COLUMNS 

500 

max  number  of  columns 

ELEMENTS 

1500 

max  number  of  nonzero 
matrix  elements 

INPUT  FILE 

31 

defines  input  MPS  file 

ITERATIONS 

500 

maximum  number  of  iterations 

SOLUTION 

END  SPECS 

yes 

go  to  a  solution. 
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The  input  MPS  file  (labeled  31  in  SPECS)  will  vary  from 
problem  Co  problem.  Ic  is  uniquely  specified  given  any  Markov 
coherent  system  repair  model.  This  is  done  by  choosing  values 
for  each  of  the  following  parameters: 

States:  n  -  number  of  components 

i  =  number  of  degradation  states/component 

Iicj  =  indicator  function  indicating  whether  one 
has  an  identical  component,  k-of-n  system 
or  not 

Repair :  s  *  number  of  servers 

a  >_  0  =  mean  repair  time 

Components:  *  mean  holding  time  of  component  i  in 

degradation  state  d 

1<  i<n,  0  <  d  <  *  -  1 
Wi 

System  Type:  -  list  mincut  sets 

Penalty  Cost:  p  ^  0  for  system  failure 

Fixed  Charge :  L  0  per  repair  decision 
d 

Repair  Costs:  K  to  repair  component  i  when  in  degradation 
state  d,  1  ±  d  <_  i 
1  <_  i  n 

Objective:  V  ^  or  (minimize) 

For  large  problems  which  might  need  to  be  tested  many  times, 
it  would  be  worth  writing  a  Fortran  program  to  produce  the  MPS 
input  file  given  the  above  model  parameters.  However,  for  the 
relatively  small  problems  to  be  tested,  it  is  easier  to  create  a 
new  MPS  file  directly  for  each  separate  example. 
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Output  from  MINOS  includes: 

(1)  a  listing  of  iterations  to  solution 

(2)  Rows  section:  indicates  the  numerical  value  taken  on  by 
various  rows  at  optimality,  including  whether  or  not 
the  row  is  at  its  upper  limit  (in  the  case  of  <_ 
constraints  in  problem  (D)).  Also  the  optimal  value 

of  the  i^*1  dual  problem  variable  corresponding  to 
the  iC^  row  is  listed. 

(3)  Columns  section:  gives  optimal  values  of  variables 
and  the  reduced  costs  for  each,  i.e.,  the  coefficient 
of  variable  x^  in  the  objective  row  of  the  optimal 
simplex  tableau. 

Given  information  on  the  input,  output  and  working  efficiency 
of  MINOS,  the  question  remains  as  whether  to  use  the  LP  (P)  or  (D) 
to  solve  the  problem.  One  immediate  point  favoring  (D)  is  that 
the  u^  variables  are  given  as  output  as  well  as  the  x^  while  only 
the  x^' s  appear  in  the  output  for  (P).  However,  the  efficiency  of 
MINOS  depends  the  most  on  the  number  of  rows  and  less  so  on  the 
columns.  This  favors  use  of  (P) ,  which  has  many  fewer  rows  than 
columns,  in  cases  where  only  V  ^  optimality  is  desired  or  the 
problem  (D)  would  have  too  many  rows  to  be  efficiently  run  under 
MINOS.  The  advantage  of  problem  (D)  is  in  cases  where  a  Vq 
optimal  solution  is  sought.  To  determine  the  restricted  decision 
space  A'  for  the  altered  Markov  decision  problem,  one  must  know 


which  of  the  rows  (i,  k)  in  (D)  are  at  their  upper  bound  at 
optimality.  This  information  appears  directly  in  the  rows  section 
output  of  problem  (D) ,  while  for  (P) ,  these  comparisons  would 
have  to  be  made  separately  in  addition  to  the  fact  that  values 
of  u  would  have  to  be  solved  for  on  the  side  using  a  system 
of  N  linear  equations  using  dual  activity  values  given  in  "rows" 
section.  In  summary,  a  rule  of  thumb  might  be 
-  if  small  problem  and  want 


r  i  only  ■*> 

use  (P) 

r  ^  and  Vq  -*■ 

use  (D) 

-  if  large  problem  and  want  to  use 

MINOS  -*•  use  (P)  if  (D)  has  too  many 

rows  for  MINOS 

or 

if  want  (D)  to  be  used,  then 
recommend  using  some  other  code. 

This  rule  will  be  followed  in  the  small  test  problems  presented  in 

Section  U.  It  should  be  noted  that  for  coherent  system  repair 

model  type  Markov  decision  chains  where  optimality  specifies 

a  unique  policy  subject  to  an  infinitesimal  change  in  one  of  the 

cost  or  lifetime  model  parameters  and  the  optimal  ergodic  chain 

is  always  irreducible,  it  will  frequently  happen  that  the  only 

* 

policy  in  &'  is  the  5  determined  by  solving  the  initial  LP. 

In  such  cases,  only  one  LP  need  be  solved  to  get  a  complete 
optimal  solution.  This  occurs  in  Examples  6.2  and  6.3  of  the  next 


section. 


In  solving  certain  problems  by  LP,  it  may  happen  that 


t*  -  u±  -  1  P^j  Uj  +  v*g  -  r^  <  0  v  k,  some  i  . 

If  so,  then  slight  modifications  in  the  state  and  decision  spaces 
of  the  altered  Markov  decision  problem  must  be  made.  Let 

•  (k  6  4^  :  t^  «  0}.  If  $  v  i,  then  there  are  no  problems, 

define  A’  as  before.  Suppose  ■  <p  for  some  states  i.  Let 

ft  =  {i  :  4  $}•  Notice  that  i  ft  can  occur  only  if  i  is 

transient  under  all  gain  optimal  policies,  a  condition  which  is 
usually  satisfied  by  all  transient  states  in  a  coherent  system 
repair  model,  unless  there  are  ties  among  some  of  the  V°  values. 
Modify  ft  and  B^  using  the  following  algorithm: 

Algorithm  1: 

(1)  Look  for  some  i  E  ft  and  k  G  B^  :  p^  >  0  for  some 

j  £  ft.  If  such  a  pair  exists,  go  to  step  (2),  otherwise  stop. 

(2)  Delete  k  from  B^.  If  doing  so  renders  B^  empty, 

delete  1  from  ft.  Go  to  step  (1).  □ 

Now,  let  the  state  space  in  the  altered  program  be  restricted 
to  ft  (using  its  terminal  definition  from  the  above  algorithm)  and 
let  decisions  be  restricted  to  the  terminal  definitions  of  B^.  The 
altered  Markov  decision  problem  will  now  find  a  Vq  optimal  policy 
which  is  Independent  of  states  i  ^  ft.  Examples  of  states  i  ^  ft 
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in  coherent  system  repair  models  would  be  ones  which  are  inaccessible 
from  the  starting  state  i^  «  all  components  working  under  a  certain 
policy.  The  algorithm  will  never  remove  decisions  corresponding  to 
ergodic  states. 

There  is  one  other  difficulty  which  can  arise.  The  terminal 
definition  of  ft  using  Algorithm  1  may  delete  some  states  that 
are  accessible  from  the  initial  state,  an  unacceptable  situation 
since  in  a  coherent  system  repair  model  it  is  desired  to  specify  an 
optimal  decision  in  every  state  that  is  accessible  from  the  state 
with  all  components  working.  Such  a  situation  may  be  corrected 
by: 

(1)  Run  the  LP  (D)  once  and  note  which  primal  variables 
are  >  0  to  get  the  optimal  decisions  in  the  ergodic 
states.  If  Bj  ”  <t>  for  any  i  or  the  terminal 
definition  of  ft  in  Algorithm  1  includes  all  states 
accessible  from  the  initial  state,  then  proceed  as 
described  previously.  If  not,  then 

(2)  Rerun  (D)  but  first  delete  rows  corresponding  to 
nonoptimal  decisions  in  ergodic  states.  This  should 
now  produce  a  ft  which  Includes  all  i  accessible 
from  {1,  ...,  n}.  Now  proceed  as  before,  (see 
Example  6.3;  k  ■  1  p  ■  3,  for  example). 

If  policy  improvement  is  used  instead  of  linear  programming, 

Ic 

such  difficulties  never  occur  for  t^  *  0  for  some  k  for  each  i. 
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4.  Some  TesC  Results 


In  this  section  some  test  results  for  some  small  coherent 
system  repair  models  will  be  presented.  First,  Example  6.2  was 
looked  at  to  make  sure  the  program  was  working,  as  its  optimal 
policy  can  be  computed  using  theory  from  Chapter  II.  Only  the 
relevant  output  is  presented.  See  Figures  6.1  and  6.2,  6.3  at 
the  end  of  the  chapter  for  sample  input  and  output  from  MINOS. 

Example  6.2:  Basic  Model,  n  *  2,  parallel  system. 


Kx-2  K2-6  p  -  1 

W1  ■  1  u2  m  2  L-0 


From  theory 


■  min 


} 


80, 


-  min  {3,  3.5,  5}  -  3 
<?  -  {0}  6*  -  Rx 


so  5^  ■  A  ■  A  is  optimal. 


optimal  policy: 


12  0 


Results:  (using  (P)  or  (D)  -  both  were  tested). 

a 

primal  variables:  x^2  “  0 

A 

at  optimality  x^  ■  0 

r2 

xx  -  0 

dual  variables:  u^2  *  -7  u2  *  -6  g  -  3 

at  optimality  Uj  *  -3  u^  =  0 

rows  at  upper  12,  A  2,  A 

bound  in  (D) : 

1,  A  0,  R1 

conclusions: 

(1)  8  -  {0}  5*  -  Rx 

(2)  A'  ”  {12, A;  1,A;  2, A,  0,  R^}  *  single  policy 

so  AAR^  is  Vq  optimal  (no  need  to  do  second  LP) 

(3)  Optimal  gain  is  3.0 

(4)  These  results  agree  with  the  theory.  □ 

Now,  consider  the  following  problem  using  the  Basic  )fodel 
which  is  not  so  trivial: 
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Kl  =  1 


K2  -  1.55 


K3  -  3.2 


*1  =  3 


X2  »  2 


X1  “  1 


p  varies 


k  =  type  of  system  *  1  or  2  of  n. 


Theoretical  results  for  V 


1,  p  -  1  V_1  =  minU^K.  +  p),  1  =  1,  2,  3;  X^  +  X^} 


X3(K3  +  p)  ■  4.2  &  -  {0}  *  R3 


k  -  1,  P  =  2  V_x  -  X3(K3  +  p)  =  5.2 


k  »  1,  p  =  3  V_1  =  XXKX  +  X2K2  -  6.1 


k  -  1,  p  -  10  V_x  -  X^  +  X2K2  -  6.1 


{0}  5q  -  R3 


{1,  2}  &1  =  R2 
62  =  R1 


k  -  2,  p  -  2  V_3  -  min  +  X^  +  X^, 


+  P)  +  Vjttj  +  p)i?Jj}  1, j  e  {1,2,3} 


X^  +  X2K2  +  X3K3  »  3  +  3.1  +  3.2  -  9.3 
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so, 


«?-  {12  13 


23) 


* 

*12  “ 


13 


23 


k  -  2,  p  -  .9  V_1  -  >2(K2  +  p)  +  X3(K3  +  p)  =  9.1 

k  -  2,  p  -  .5  V_1  -  A2(K2  +  p)  +  A3(K3  +  p)  -  7.8 

so,  $  -  {2,  3}  6*  -  R3  6*  -  R2 


Computational  results: 


k  -  1,  P  -  1 


primal  variables: 


1.0 


otherwise 


dual  rows  at 
upper  limit:  123, A 

12,  A 

13,  A 


23, A  3, A 

1»A  0,R3 

2, A 


conclusions: 

(1)  <?-  {o},  5*  *  R3  (agrees  with  theory) 


(2)  optimal  to  do  nothing  in  other  states  -  no 

need  for  second  LP,  i.e.,  "A"  in  states  1,2,3. 


Notes: 

(1)  In  all  of  the  variations  in  Example  6.2,  to  save  time, 
decision  variables  for  states  s  :  |s|  >  k  were  fixed  to  "A" 
since  known  to  be  so  by  Chapter  II  results. 
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(2)  The  dual  LP  was  used  In  all  cases  here  to  obtain 


solutions. 


2,  k  -  1 


primal  variables: 

R3 

xo  =  1-° 

k  n 

=  0,  ( 

dual  rows  at 

upper  limit: 

123, A 

23, A  3, A 

12, A 

i,r3  o,r. 

13, A 

2,R3 

otherwise 


conclusions : 

(1)  <^=  {0},  6q  =  (agrees  with  theory) 

(2)  optimal  policy  in  states  1,  2,  3: 


1 

2 

3 


R 

R 


3 

3 


:  A 


no  need  for  second  LP. 


P  =  3,  k  =  1 

primal  variables: 


2.0 


3.0 


k 


0,  otherwise 


i st 

1  run 

of  (D) 

run  (D) 

with 

1.«.6  *  r2 

2,6,6  t  Rx 

dual  rows  at 

123, A 

l.R, 

123, A 

1>R2 

upper  limit: 

12, A 

1,R3 

12, A 

2,R 

rows  eliminated 

B0  "  ♦ 

13, A 

23, A 

2, RX 

3, A 

13, A 

23, A 

3, A 

°,r12 

0  -  {12,1,2} 

after 

use  ALG  1. 

Okay. 
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conclusions: 


(1)  S  -  {1  2}  6*  -  R2,  6*  - 

(2)  optimal  decisions  in  states  3,  0: 

3  :  A  0  :  R^2 

no  second  LP  necessary. 

p  =  10,  k  *  1 


R2 

primal  variables: 

xx  -  2.0 

R1 

x2  =  3.0 

x^  =  0,  otherwise 

dual  rows  at 

upper  limit: 

123, A  13, A 

3, A  Bfl  -  * 

12, A  1,R2 

3,RX 

23, A  2,R1 

0  =  {all  states  but  0} 

after  Algorithm  1. 

okay,  since 

0  inaccessible  from  123. 

*  * 

conclusions : 

(1)  <?  =  (1  2} 

61  =  R2  62  =  R1 

(2)  optimal  decisions  in  tranient  states: 

3  :  R1 

will  never  enter  state  0 
so  can  ignore. 

■  .5,  k  =  2 

primal  variables: 

R3 

x2J  =  1.0 

r2 

=  2.0 

=  0,  otherwise 

dual  rows  at 

upper  limit: 

123, A  13, A 

1,R3  3,R2 

12, A  23, A 

2,R3 
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conclusions : 


(1) 

(2) 


£  =  (2  3}  6*  =  R3  6*  =  R2 

optimal  decisions  in  transient  states: 


Note:  In  the  input  of  data,  one  must  be  very  careful  to  be  accurate 
and  carry  at  least  4-5  decimal  points  in  determining  the  correct 
Vq  optimal  solution.  The  set  of  dual  rows  which  are  at  their  upper 
limit  can  be  very  sensitive  to  small  changes  in  the  parameters. 
Realistically,  if  two  policies  are  that  close,  using  either  would 
be  optimal  and  the  easier  one  to  implement,  practically  speaking,  could 
be  chosen.  If  only  interested  in  V_^,  the  such  accuracy  is  not  as 
important. 


p  =  .9,  k  =  2 


primal  variables: 


1.0 


2.0 


otherwise 


dual  rows  at 

upper  limit:  123, A  23, A  3,R2 


12 » R3  1,R23 

13, R2  2,R3 


conclusion: 


<?=  (2,  3} 


transient  states: 


R 


23 
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primal  variables: 


x12  -  1.0 


x^  ■  0,  otherwise 


x13  "  2'° 
R1 

x23  *  3<0 


dual  rows  at 


upper  limit: 

123, A 

23, A 

12,  R3 

23,  Rx 

13,  Rj 

1,R23 

conclusions: 

(1)  <f-  {12, 
R3 

13,  23} 
R2  R1 

2,r3  b3  -  4> 

Q  *  {123,12,13,23} 
after  ALG  1,  okay. 

states  1,2,3  Inaccessible. 


(2)  6^,  62,  63  never  needed  since  inaccessible  given  set 

of  ergodic  states.  These  values  actually  could  be 
obtained  by  dropping  rows  corresponding  to  nonoptimal 
decisions  in  ergodic  states  and  rerunning  the  problem. 


Now,  consider  a  noninstantaneous  repair  example,  for  which 
only  the  V_^  solution  is  sought.  In  this  case,  problem  (P)  is 
used  for  determining  the  solution. 


Example  6.4:  Noninstantaneous  repair  model,  n  *  4 

L  *  varies  K  =  1  o  =  1  parallel  system 

p  ■  varies  p  ■  1  1*0 


s  ■  number  of  servers  varies 


Theoretical  results: 


Unknown,  except  in  s  *  1  cases  with  L  -  0. 
The  following  parameter  values  were  tested: 


L 


0  1.4  9.5  49.5  72.5  74.5  76.5  78.5  89.5  99.5 


s  -  4.  L  -  0 


Results : 


s  ■  4,  p  *  1  (varying  L) 


optimal  ergodic  chain 
(optimal  policy) 


0,0 


1,0 

A 


[R<0)] 


1.4 

9.5 

49.5 

72.5 

74.5 

76.5 


R(0) 

R<0) 

R<°> 

R<°> 

R(0) 

R<°> 


78.5 


4,0  3,0  2,1  2,2  1,0  1,1  1,2  1,3  0,0  0,1  0,2  0,3 

A  A  A  A  A  A  A  A  R^R^RjA, 


89.5 

99.5 


A  policy  which  says  do  nothing  until  the  system  fails,  then 
repair  all  failed  items  possible  subject  to  server  availability. 

(same  as  L  »  78.5) 
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» 

t 


Clearly,  the  L  "  78.5  policy  should  be  optimal  for  all  L  values 
large  enough.  The  fact  that  the  mean  repair  time  Is  large 
(“  mean  component  lifetimes)  explains  why  the  fixed  charge  must 
get  so  large  to  have  any  effect  on  optimal  policy,  (large  o  favors 
repair  only  when  necessary) . 


s=4,  L  ■  0  results 


p 

optimal  policy 

1 

1,0  0,0 

A  Rx  (R(0)) 

5 

2,0  1,0  0,1 

A  Rx  A  (R(1)) 

7 

3,0  2,0  1,0  1,1  0,1 

A  R1  Rx  A  A  (R(2)) 

10 

4,0  3,0  3,1  2,1  2,2  1,2  0,3 

A  Rx  A  Rx  A  A 

20 

same  as  p  =  10 

requires  use  of 
more  than  one  server 


The  p  =  10  case  is  another  example  of  a  noninstantaneous 
repair  model  for  which  it  is  optimal  to  use  more  than  one  server 
at  a  time  (unlike  instantaneous  repair). 

Clearly,  if  the  s  =  4  optimal  policy  never  uses  more  than 
s'  <  4  servers,  then  the  s  =  s’  optimal  policy  will  be  the  same 
as  for  s  ■  4.  Thus,  the  p  ■  20  case  is  chosen  as  the  one  to 
vary  s  in. 
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“  20, 

.  L  -  0.  s 

*  1.  2, 

3,  4 

results 

s 

V  optimal  policy 

4 

4,0  3,0 

A  R1 

3,1 

A 

2,1 

R1 

2,2 

A 

1,2 

R1 

0,3 

A 

(uses  3  servers) 

3 

(same  as 

s  ■  4) 

2 

4,0'  3,0 

A 

3,1 

A 

2,1 

R1 

1,1 

R1 

1,2 

A 

0,2 

A 

(uses  2  servers) 

1 

4,0  3,0 

A  Rx 

2,0 

R1 

2,1 

A 

1,0 

R1 

1,1 

A 

0,1 

A 

/  uses\ 

V  1  server  / 

The  previous  examples  are  Intended  only  to  give  a  sampling 
of  the  kinds  of  new  results  obtainable  computationally.  Further 
experimentation  could  lead  to  formulation  of  new  theorems  or 
counterexamples  to  certain  conjectures  about  optimal  policy  forms. 
Most  importantly  of  all,  these  techniques  could  be  Implemented  to 
solve  a  specific  real-world  problem  which  could  be  formulated  as  a 
coherent  system  repair  model.  Sample  input  and  output  from 
Example  6.2  are  shown  in  Figures  6.1,  6.2,  6.3. 
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FIGURE  6.1  -  MPS  Input  Data  File  for  Example  6.2  (Dual  LP) 


U  0  100 

NAME 

EX1U 

00200 

ROWS 

00300 

M  OBJ 

00400 

L  XI  2ft 

00500 

L  XI  ft 

00600 

L  XI R2 

00700 

L  X2fi 

00800 

L  X2R1 

00900 

L  X0R1 

01000 

L  X0R2 

0  1  1  0  0 

L  X0R12 

01200 

COLUMNS 

01300 

U12 

X12fl 

1 . 0 

0  1 400 

U1 

X12H 

-  .3333 

XI  H 

1.0 

01500 

U1 

X  1  R2 

•  666? 

X2R 1 

-  .3333 

0  1600 

U1 

X0R12 

-  .3333 

01700 

U2 

X 1 2rt 

-  .  6667 

X 1 R2 

-  . 6667 

01800 

U2 

X2H 

1 .0 

X2R1 

.3333 

01900 

U2 

X0R12 

-  .  6667 

02000 

UO 

XI H 

-1.0 

X2H 

-1.0 

02100 

0 

OBJ 

-1.0 

X12R 

.  6667 

02200 

G 

Xlfl 

1.0 

XI  R2 

.  6667 

0230  u 

G 

X2ft 

2.0 

X2R 1 

.  6667 

02400 

G 

X0R1 

1 .0 

X0R2 

2.0 

02500 

G 

X0R12 

. 6667$ 

#b 

CEX1U. 

DrtT .  1  J 

*12500 

02600 

RHS 

02700 

RHS 

X1R2 

6.  0 

X2R1 

2.0 

02800 

RHS 

X0R1 

3.0 

X0R2 

7.0 

02900 

RHS 

X0RI2 

9.  0 

030  00 

BOUNDS 

03100 

PR  BHD 

U12 

03200 

FR  BND 

U1 

03300 

FR  BND 

U2 

0  340  0 

FR  BND 

UO 

03500 

FR  BND 

G 

03600 

ENDflTft* 

8  decisions  ■*  9  rows  (counting  objective) 
4  states  ■*  5  columns 

The  primal  would  have  5  rows  and  9  columns. 
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FIGURE  6.2  -  MINOS  Row  and  p  ■  1  Colua 
Output  for  Example  6.2  (Dual, 


l?3A6 

NIIHREP 

...Pf>H..  ** 

.  .  .Pf  TKltTV.  . . 

Si  sr» 

prTTMlTV 

. .i»«ee  :nit. 

. .UPPFP 

i  r*tr.  . 

mini  i 

Or  t  f»ir  TV 

.  .  1 

1  «MI 

1 

V 

npj 

8$ 

-3.P0006 

3.PP000 

HOWE 

hOhf 

-1 .00000 

I 

1?RPA 

8 

VI?P 

ML 

A. 00666 

6.P0000 

NONE 

0. 0006A 

A.  00000 

o 

12**9 

9 

VIP 

ML 

A  .0  0066 

6.C6006 

NONE 

A. 6000P 

A  .  00  600 

3 

1  70AA 

1  A 

VIP? 

PS 

* . 00020 

1 .99990 

NONE 

6.CM9A 

0.00060 

u 

HIM 

1 I 

V?P 

ML 

0. PAA0A 

0.00600 

NONE 

A. 06666 

0.00000 

5 

I72O0 

!* 

V?PI 

RS 

1 .00020 

p . 999RA 

NONE 

2.0006A 

0  .  A  A  A  0  A 

6 

IJ30O 

1  7 

:api 

III 

7.00000 

0.06000 

NONE 

3. 0000 A 

-|  .00000 

7 

t  ?*•«<! 

1  * 

VAP? 

RS 

6.00600 

1  .00060 

NONE 

7.00006 

A  .OAA06 

R 

I73A0 
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CHAPTER  VII 


CONCLUSIONS,  APPLICATIONS  AND  EXTENSIONS 

1.  Summary  of  Model  Results  -  Chapters  II-V 

This  chapter  will  present  a  summary  of  results  from  previous 
chapters  in  this  section.  Following  that  in  Section  2,  two  basic 
theorems  which  relate  to  all  coherent  system  repair  models  are 
drawn  out  from  the  results.  Under  what  parameter  assumptions  they 
do  or  do  not  apply  is  discussed.  Section  3  looks  into  some 
conclusions  one  might  draw  from  Sections  1  and  2  results  about  how 
changes  in  various  model  parameters  affect  the  optimal  policy  or  the 
minimum  cost  per  unit  time  attainable.  Possible  applications  of 
these  models  or  the  results  of  such  are  discussed  in  Section  4.  A 
listing  of  possible  model  extensions  and  topics  for  future  research 
concludes  the  chapter  and  thesis. 

The  summary  of  previous  results  along  with  references  to  the 
chapter  section,  and  theorem  from  which  each  was  drawn  will  be 
presented  as  a  series  of  tables.  This  allows  for  a  clear  and  concise 
categorization  of  the  numerous  items. 

Classification  will  be  done  according  to  model  type  and 
particular  parameter  assumptions  made  for  it.  Table  7.1  summarizes 
the  categories: 
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TABLE  7 . 1  -  Categorization  of  Results  for 
Coherent  System  Repair  Models 


I.  Basic  Model 


Found  in 


A.  system  type:  general  coherent,  +  series' 


objective:  V 


-1 


Table  7.2 


B.  system  type:  k-of-n,  k  +  n  (series) 
objective:  V 


Table  7.3 


C.  system  type:  k-of-n,  k  f  n  (series) 
objective:  V 


Table  7.4 


II.  Degradation  Model 


A.  system  type: 
objective: 


series  or  parallel 


III.  Noninstantaneous  Repair  Model 


A.  system  type: 
objective: 


k-of-n,  identical 
components  (nonseries) 


Table  7.5 


Table  7.6 


Tables  7.2  7.6  will  comprise  the  rest  of  this  section. 
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TABLE  7.2  -  Basic  Model,  General  Coherent 
System,  V  Results 


L  =  0 

No 

fixed 

charge 


Chapter 


[Section iTheorem(s) 


For  any  policy  6,  V  ^  can  be  written 


II 


as  a  convex  combination  of  certain 
where 


-1 


Yg  are  policies  which  never  re¬ 
pair  more  than  one  unit  simultaneously. 


Never  repair  more  than  one  unit  at  a  j 
time  in  an  ergodic  state. 

V_^  optimal  policy:  keep  a  certain 

subset  of  components  working  -  repair 

any  that  fail  immediately,  s  determined 

by  s  =  argmin  V  where 
tC{l,...,n} 


II 


II 


V  = 
t 


K. 

l  ~  + 

i4t  yi 


I 

iet 

P 


K.+p 

i 

yi 


1 


2.1,  2.2 


2.3 


2.4 


1 

If  one  can  break  the  system  down  into 

independent  subsystems  in  series,  the  1 

VI  ! 

3 

6.3 

i 

optimal  system  V  .  is  the  sum  of  the  ; 

» g  “’■I-  i 

optimal  V  for  each  subsystem.  1 

I 

L  >  0 

6 

Still  have  V  written  as  a  convex 

combination  of  certain  quantities  but 

j 

fixed 

’s  1 

these  are  now  not  all  V  of  simpler 

II 

3 

2'5 

charge 

—  1 

policies.  j 
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TABLE  7.3  -  Basic  Model,  k-of-n  System. 


-1 


Results  (k  ^  n  series) 


All  results  from  Table  7.2  for  ^  >  ^  plus: 
nonidentical  components  n  identical  components 


V  optimal  policy  form: 

Keep  cheapest  set  of  k-1 
or  k  components  working. 

Ch.  Ill 
Sec.  2 
Ex . 2 . 4 

Parallel  case:  if  have 
sufficiently  large  spread 

of  X^K^'s  compared  to 

X^'s,  then  V  optimal 

policy  will  repair  the 
items  with  cheapest  X^K^, 

no  matter  what  p  is. 

Ch. Ill 
Sec.  2 
Lem  3.12 

Policy  form:  too  compli¬ 
cated  in  general  for  a 
nice  result. 

Ch.  II 
Sec.  3 

If  p=0;  do  nothing  until 
system  fails 

Ch.  II 
Sec.  3 

identical  components 

L  >  0 


Optimal  form:  do  nothing 
until  k  units  left  working) 
Repair  one  unit  in  k  or 
k-1,  depending  on  p. 


Policy  form:  do  nothing 
until  k  items  left  working) 

-among  policies  which  re¬ 
pair  when  k  left,  R 


optimal  f 

<  W*> 


k+1 


i-1 
(«.-!)  <  L/K 


-among  policies  which  re¬ 
pair  when  k-1  left,  R^ 

optimal  **  f^()l-l)  < 


<  fk(D 


L  >  0 


If  n1<n2, 


number  of  com¬ 


ponents  repaired  in  an 
joptimal  policy  for  k-of-n^ 

is  number  repaired  for 
a  k-of-n.. .  If  number 
repaired  For  n^  is  less| 

than  all  possible,  then 
the  number  repaired  for 
n„  is  the  same. 


Ch. 

Sec 

Thm 


II 
,  3 

2.ai 


-optimal  policy  is  the 
better  of  the  previous  two| 
where 

fk(o-a*)g+— 


If  k^<k2»  then  an  optimal 
k2-of-n  policy  will  re¬ 
pair  more  components  than 
a  k^-of-n. 

If  n,k-«°,  n-k  fixed,  an 
optimal  policy  will  be  to 
repair  all  failed  compo¬ 
nents  any  time  repair  is 
undertaken. 


Ch.  II 
Sec .  2 
special 
case  - 
Ex.  2.4 


Ch.  II 
Sec.  3 
Thm  2.6) 


Ch.  II 
Sec.  3 
Thm  2 . 7| 


Ch.  II 
Sec.  3 
Thm  2.9! 
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FT 


TABLE  7.4  -  Basic  Model,  Results,  k-of-n  System 


nonidentical  components 


Do  nothing  until  k  units 

Ch.  II 

left  working. 

Sec .  3  & 
Ch  .111 , 1 
Thm.  3.3 

Parallel  system;  L  =  0: 

Ch.  Ill 

never  repair  up  to  more 

Sec.  1 

than  two  components  working 
simultaneously. 

Thm.  3.4 

k 

y  x.  k. 

£  i  i 

If  p<  r  r-V-,  then 

1  k 

Ch.  Ill 
Sec .  1 

never  repair  up  to  more  than 
k  +  1  units  working 
simultaneously. 

Lem .3.5 

Parallel  case;  L  =  0 

i  J 

and  >  Kj  ^  never  repair 

Ch.  Ill 
Sec.  2 
Lem. 3.6 

unit  i . 

Parallel  case:  restrictions 

Ch.  Ill 

on  possible  optimal  policies 

Sec.  2 

given  V_L  =  X^Oc^+p),  any  £. 

LS.  3.7, 3.8 

Th.  3.9 
Cor .3.10 

Parallel  case:  restrictions 

Ch.  Ill 

on  optimal  policies  given  a 

Sec.  2 

specific  l  in  V  ^  = 

VK«  +  »■ 

Lem. 3.11 

identical  components 


NOT  NEEDED 


V_^  gives 
unique  optimal 


decision  in 


each  accessible 


state 
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TABLE  7.5  -  Degradation  Model  Results  (and  Conjectures) 


nonidentical  components 

identical  components  1 

Optimal  policy:  Let 

Ch.  IV 

Optimal  policy:  When 

Ch.  IV 

series 

system 

i*"*1  component  get  down 
to  a  certain  state  of 
degradation,  say  - 

then  repair,  where 

is  determined  by  the 

Sec.  2 
Th.  4.1 
Lem .4.2 
Lem. 4. 3 

any  component  reaches 

state  of  degrada¬ 
tion,  repair  it. 

[get  using  V  only]. 

Sec.  2 
Th.  4.1 
Lem. 4 .2 
Lem. 4 . 3 

K's  and  X's  [get  using 
V  .  only]. 

Conjectures : 

(These  results  still 
need  to  be  verified) 

parallel 

system 

Too  complicated  to 
analyze  theoretically. 

Never  optimal  to  re¬ 
pair  >  1  unit  at  a 
time  in  an  ergodic 
state. 

Ch.  IV 
Sec.  3 

Never  repair  until 
one  unit  or  less 
working. 

Ch.  iv 
Sec.  3 

general 

system 

Can  modify  degradation 
model  slightly  to  get 
an  extension  of  the 
Basic  Model  allowing 
for  Erlang  component 
lifetimes. 

Ch.  Ill 
Sec.  4 

V  optimal  solution 

to  M,  . .  same  as 
(n,a) 

that  to  !■  >  b 

n  >  2,  where  M  „  is 
—  n,i 

a  Degradation  Model 

with  n  components, 

l  degradation  states. 

Ch.  IV 
Sec.  3 
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TABLE  7.6  -  Nonins t an taneous  Repair  Model  Results 


I  Chap-  Sec-i 
ter  kionJ  Theorem 


V_^  optimal  policy  form:  repair  whenever  the 

server  is  free  and  the  number  of  working 
components  is  below  a  certain  number,  say  j . 

(Call  this  policy  R^). 

The  optimal  policy  is 


2  Th.  5.1 
Lem. 5. 2 
Th.  5.3 


R  j  ,  if  u(j)  <  <  u(j  +  1) 

(k-1)  PP 

R  ’  lf  0<f^UO<U<k> 
where 


L  -  0 


z  =  x.  h±k  =  +  oz 

Z1  j+1  j+1  +  °Zj 


t  -  !  f  -  .  k  >  2 

1  K  pK  (y+(k-l)o) 

U(J) _ -  r^-^i 

J  oJ-i(J-i),  Lj  j+iJ 


Relationship  of  optimal  policy  to  k,  f  : 
number  of  components  repaired  could 
increase  or  decrease  with  k,  depending 
on  p ,  a . 


2  Lem. 5. 4 
Lem. 5 . 5 


>  ^  Not  true  that  no  fixed  charge  means  never 
repair  more  than  one  unit  at  a  time. 

multiple  a  p0iiCy  which  uses  s’  servers  at  most 

servers  is  °Pt^ma^  for  a  ™odel  which  has  s  >  s', 
then  it  is  optimal  for  the  model  with  any 
s'  <  t  <  s. 


3  Ex.  5.5 

4  Ex.  6.3 

3  Lem. 5. 6 
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2 .  Two  Basic  Theorems 


Section  1  presented  a  summary  of  results  obtained  in 
Chapters  II  through  V  for  three  different  categories  of  coherent 
system  repair  models.  Due  to  significant  differences  in  state 
space,  decision  space,  and  component  lifetime  distributions,  results 
for  each  model  had  to  be  obtained  separately  and  in  general,  optimal 
policy  forms  are  quite  different.  Even  results  of  similar  type  had 
to  be  proved  in  a  different  fashion  for  each  model.  Thus,  the 
allocation  of  separate  chapters  to  each  model,  even  though  all  came 
under  the  heading  of  a  coherent  system  repair  model. 

Despite  these  differences  in  model  structure  and,  thus, 
methods  of  obtaining  results,  one  would  expect  similarities  between 
many  of  the  models  since  they  are  all  modeling  the  same  type  of 
activity  -  the  maintenance  of  a  deteriorating  system  of  components. 
Two  basic  policy  "types"  appear  throughout.  These  are: 

(1)  If  one  has  a  k-of-n  system  and  instantaneous  repair, 
then  never  repair  until  there  are  k  units  left 

in  the  system. 

(2)  If  there  is  no  fixed  charge  for  repair  (only  component 
and  labor  costs) ,  then  one  never  repairs  more  than  one 
unit  at  a  time  in  an  optimal  policy  in  an  ergodic  state. 

The  applicability  of  statements  (1)  and  (2)  to  the  various  coherent 
system  repair  models  discussed  up  to  now  appears  in  Tables  7.7  and 
7.8. 
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TABLE  7 . 7  -  Application  of  Statement: 


"If  have  a  k-of-n  system  with  Instantaneous  repair,  then  never 
repair  when  more  than  k  units  in  the  system  are  up" 


to  various  coherent  system  repair  models. 


Basic  Model  (k-of-n) 


True 

True 

True 

True 

Notes 


for  proof 
see  Theorems 
2.3,  3.3 


Degradation  Model  (L  =  0) 
series  parallel 


identical 

components 

True 

conjecture 

True 

(true  n  =  2, 
l  -  2) 

nonidentical 

conjecture  to  be  true 

components, 

L  >  0 

(future  research) 

series  is  obvious 


Noninstantaneous  Repair  Model  -  False  see  Lemma  4.1  (s  *  1) 

for  counterexample. 
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TABLE  7.8  -  Application  of  Statement: 


"If  there  is  no  fixed  charge  for  repair,  then  one  never  repairs 
more  than  one  unit  at  a  time  in  an  optimal  policy  in  an  ergodic 
state" 


to  various  coherent  system  repair  models 


Notes 


Theorem  2 . 3 

also,  false  if  extended 
to  Erlang  case 
(see  Example  A. 3) 


Degradation  Model 
nonidentical  identical 
components  components 


Series 

True 

True 

conjecture 

conjecture 

true 

true 

parallel 

(future 

true 

research) 

n=2,  1=2 
_ 

Series  - 

Lemmas  A. 2,  A. 3 
plus  Theorem  A.l 


Nonins tantaneous  Repair  Model 


False 


see  Examples  5.5,  6. A 
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Statement  (1)  is  intuitive,  although  nontrivial  to  prove  in 
many  cases.  It  applies  to  all  instantaneous  repair  models  considered 
and  I  suspect  it  applies  to  others  as  well.  The  second  statement  is 
a  much  more  interesting  result.  It  seeks  conditions  on  model 
parameters  under  which  if  there  is  no  fixed  cost  per  repair  decision, 
then  no  more  than  one  unit  ever  will  Is  repaired  at  one  time  once 
the  set  of  ergodic  states  has  been  reached. 

Given  statement  (2)  concerning  the  ergodic  states,  the  next 
natural  question  to  ask  is,  what  about  the  transient  ones,  i.e., 
does  a  similar  result  to  (2)  hold  in  states  not  in  <P?  The  answer 
is  definitely  no  if  all  transient  states  are  considered.  Even  in 
the  simplest  parallel  case,  it  can  be  optimal  to  repair  two  units 
at  once  in  state  0  if  it  happens  to  be  transient,  (for  specific 
example  see  Example  6.3,  p  «  3,  k  *  1  case).  However,  if  statement 
(2)  is  reworded  to  say: 

(3)  "If  there  is  no  fixed  charge,  then  it  is  never  optimal 
to  repair  more  components  than  are  needed  to  bring  the 
number  working  to  level 

then  it  becomes  applicable  in  certain  cases. 

If  the  system  is  parallel  and  i  «  2,  then  (3)  holds,  (see 
Table  7.4).  In  the  k-of-n  case  with  i  ■  k  +  1  and 

P  P 

p  <  — = -  ,  (3)  also  holds  (see  Table  7.4).  Whether  or  not  it 

4Xi 

P 

holds  for  any  k-of-n  system  is  still  up  in  the  air.  For  a 
general  coherent  system,  l  would  be  the  number  of  units  in  the 
minimum  path  set  which  lias  the  largest  number  of  components. 
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3.  Effects  on  and  Sensitivity  of  Models  to  Various  Parameters 

Section  2  discussed  relationships  between  various  coherent 
system  repair  models  with  respect  to  the  applicability  or  not  of 
two  basic  theorems  restricting  the  policy  space.  In  this  section 
conclusions  concerning  parameter-related  interrelationships  are 
drawn  and  these  are  compared  to  the  expected  reality  in  a  real-world 
situation.  Effects  on  the  optimal  decisions  as  well  as  on  the  minimum 
cost  per  unit  time  attainable  will  be  noted.  Parameters  will  be 
treated  in  the  following  order: 

(a)  system  type  (k-of-n,  varying  k  or  n) 

(b)  penalty  cost  for  system  breakdown,  p 

(c)  fixed  charge  for  repair,  L 

(d)  mean  repair  time,  o 

(e)  component  lifetimes  (means  or  distributions) 

(f)  objective:  when  V  ^  is  sufficient  to  determine 
optimality. 

Most  of  these  effects  will  be  intuitively  expected  ones,  supporting 

the  validity  of  the  coherent  system  repair  model. 

Consider  a  coherent  system  repair  model  for  a  k-of-n  system 

which  has  instantaneous  repair.  If  k  is  increased  to  k^,  then 

the  decision  space  is  reduced  (since  one  will  never  reach  states 

s  :  |s|  <  k.  -  1  and  S  ?  A  if  Isl  =  k,  -  1.  Meanwhile,  the 
1  s  1  1 

costs  per  decision  are  either  left  the  same  or  raised  (for 
s : | s [  *  k  -  1,  repair  cost  now  includes  p,  system  breakdown  cost). 
Thus,  the  minimum  V  ^  (or  V^)  attainable  is  larger  for  higher 
k  given  fixed  n.  Similarly,  if  k  is  fixed  and  n  is  raised. 
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costs  on  previous  decisions  remain  the  same  while  some  new  states 
and  decisions  are  added.  Thus,  the  minimum  V  ^  (V^)  attainable 
is  lower.  These  results  coincide  with  what  might  be  expected  given 
the  reliability  of  a  k-of-n  system  goes  down  as  k  increases 
and  goes  up  as  n  increases. 

The  noninstantaneous  repair  model  is  the  only  one  for  which 
this  does  not  occur.  This  behavior  is  due  to  two  facts: 

(1)  With  the  noninstantaneous  repair  model  structure, 
it  is  still  possible  to  enter  states  s  where 

|s |  <  k  -  1. 

(2)  The  penalty  cost  is  applied  upon  entering  a  state 

s  :  | s j  =  k  -  1  only.  No  p  is  incurred  in  states 

s  :  |s|  <  k  -  1.  If  p  were  incurred  in  all  states 

|s|  <_  k  -  1,  then  the  previous  argument  would  apply 

and  minimum  V would  increase  with  k. 

Lemmas  5.4  and  5.5  show  that,  for  s  =  1,  that  given 

/  2 

l+/l+4^y 

*  q  * 

k  =  - - -  ,  that  for  k  <  k  increasing  k  raises  the 

* 

minimum  V  ^  but  for  k  >  k  ,  increasing  k  lowers  the  minimum  V 

As  this  second  behavior  is  unrealistic,  one  concludes  that  the 

model  works  best  for  cases  where  a  is  enough  larger  than  y  to 
* 

make  k  _>  n.  For  large  k,  the  assumption  of  p  being  incurred 
only  at  entry  into  states  |s|  ■  k  -  1  becomes  more  unrealistic 
as  the  possibility  increases  of  being  able  to  keep  moving  from 
state  to  state  while  the  system  is  failed  while  incurring  no 
penalty.  In  such  cases,  it  might  be  better  to  assume  the  penalty 
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is  incurred  in  all  states  |s|  _<  k  -  1  even  though  the  system  has 
only  broken  down  once,  or  to  restrict  the  decision  space  to  exclude 
policies  which  have  only  "system-failed"  states  in  their  ergodic 
structure. 

Behavior  of  optimal  decisions  with  respect  to  k  are  harder 
to  characterize  due  to  the  many  other  parameter  effects  other  than 
to  say  as  k  increases  with  fixed  n,  the  number  of  components 
repaired  or  kept  working  goes  up  due  to  an  earlier  penalty 
incurrence.  For  the  Basic  Model  with  L  =  0,  the  general  V 
optimal  policy  for  a  k-of-n  system  is  to  keep  the  cheapest  set 
of  k  or  k  -  1  components  working.  In  the  L  >  0  identical 
component  Basic  Model,  Theorem  2.7  states  that  as  k  increases, 
more  components  (or  the  same  number)  will  be  repaired  in  an 

* 

optimal  policy.  Lemma  5.5  gives  the  same  result  for  k  <  k  in 

* 

the  single  server  noninstantaneous  repair  model  while  for  k  >  k 
(unrealistic  case)  the  reverse  becomes  true. 

Now,  take  the  parameter  p,  the  penalty  cost  for  system 
breakdown.  Clearly,  all  other  variables  fixed,  increasing  any 
single  cost  can  only  raise  the  possible  minimum  V  .  As  would 
be  naturally  expected,  the  higher  the  cost  of  potential  system 
failure,  the  more  likelihood  of  a  policy  being  optimal  which  avoids 
or  minimizes  the  chance  of  such  failure.  Example  6.3  illustrates 
such  policy  changes  well  for  varying  p  in  the  case  of  a  Basic 
Model  with  3  components  and  no  fixed  charge.  In  example  6.4,  a 
noninstantaneous  repair  case  shows  up.  For  the  instantaneous  repair 
models,  a  policy  can  always  be  chosen  so  as  to  avoid  the  penalty 
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entirely  and  if  p  is  high  enough,  one  of  these  will  be  chosen. 

If  p  is  zero,  there  is  no  reason  to  avoid  system  failure  and, 
indeed,  it  will  never  be  optimal  to  repair  before  such  happens  in 
any  of  the  models.  The  same  results  hold  for  noninstantaneous 
repair  except  that  there  is  no  policy  which  can  guarantee  no  system 
failures. 

The  fixed  charge  L,  has  a  clear  effect  on  policies.  The 

larger  it  gets,  the  greater  the  pressures  to  start  multiunit  repair 

simultaneously  as  opposed  to  repairing  single  units  on  different 

occasions.  At  least  in  exponential  component  lifetime  cases  if 

there  is  little  or  no  fixed  charge,  the  natural  incentive  is  to 

keep  as  few  items  working  or  under  repair  as  possible  because  such 

states  have  the  longest  holding  times,  creating  a  longer  time 

between  repairs  (costs  incurred) ,  thus  the  prevalence  of  L  =  0 

theorems  stating  under  what  conditions  one  never  will  repair  more 

than  one  unit  at  a  time.  The  identical  component  Basic  Model 

with  L  >  0  described  in  Section  2.3  and  Example  6. A  best 

illustrate  policy  behavior  as  L  increases.  Obviously,  as  L 

6 

increases,  minimum  V_^  will  increase,  with  V  increasing  the 
fastest  for  those  policies  which  undertake  repair  the  greatest 
number  of  times. 

The  mean  repair-time  effect  is  harder  to  judge.  Most 
distinguishable  of  all  differences  are  those  between  a  =  0  and 
o  >  0.  In  the  former,  the  items  in  service  do  nothing  to  affect 
system  behavior  since  time  in  service  is  always  zero.  The  decision 
maker  can  always  instantaneously  put  the  system  into  any  state  he 
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pleases.  In  contrast,  if  o  >  0,  no  matter  how  many  units  are 

sent  to  repair  at  once,  the  system  may  fail  before  repair  on  some 

or  all  is  completed.  As  a  is  increased,  repair  must  be  started 

"sooner",  i.e.,  when  fewer  components  have  failed  to  have  the  same 

effect  of  preventative  repair  of  failed  components  to  prevent 

system  failure.  The  fact  that  in  the  noninstantaneous  repair  model 

it  is  no  longer  true  that  repair  is  never  done  when  more  than  k 

units  of  a  k-of-n  system  are  working  points  out  this  effect. 

If  s  =  1  and  the  components  are  identical,  the  optimal  policy 

form  is  R^,  where  j  is  the  number  of  components  working  when 

repair  is  to  be  initiated.  Countering  this  is  the  natural  system 

tendency  to  want  to  have  as  few  items  under  repair  or  working  as 

possible  to  maximize  holding  times  between  states.  In  the  single 

server  case  of  Theorem  5.3,  as  a  gets  large,  u(k)  -*■  00  and  the 

(k-1) 

optimal  policy  approaches  R  ,  i.e.,  "it  takes  so  long  to  repair 
why  not  let  the  system  fail  anyway". 

Previously  described  parameters  help  determine  the  optimal 
number  to  repair  or  whether  to  repair  or  not.  It  is  the  mean 
lifetime  and  repair  cost  of  the  separate  components  which  determine 
the  attractiveness  of  different  components  in  deciding  which  to 
repair  over  others.  In  the  Basic  Model,  a  component's  "repair 
attractiveness"  depends  on  its  cost/mean  life  ratio,  both  with  or 


K1+P  Kt 

without  failure,  -  or  —  ,  as  is  shown  by  Theorem  2.4.  If 

ui  ui 

there  are  degradation  states,  then  the  key  indicators  of  "desirability 


of  repairing  component  i  when  in  its  j  degradation  state"  are 
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1 


n=o 


l 

i 


In  the  series  Degradation  Model,  whatever  j  produces 


.  .  .  ,  cost  to  repair  at  j  .  .  ,  .  , 

the  minimal  - —  - .  .  is  the  optimal  state  m  which  to 

expected  time  spent  in  r 

degradation  states 

0,  1,  * . . ,  j  —  1 


repair  component  i.  These  same  ratios  are  a  large  but  not  the  only 
determining  factor  in  the  parallel  case. 

The  final  remark  concerns  the  choice  of  optimality  criterion 
up  to  level  V  or  Vq.  For  most  Markov  coherent  system  repair 
models  tested,  V  was  not  sufficient  to  determine  an  optimal 
decision  in  each  state.  The  few  notable  exceptions  were: 


(1)  Basic  Model,  identical  components 

(2)  Degradation  Model,  series  case. 

V results  were  much  easier  to  obtain  and  over  an  infinite  planning 
horizon  are  the  only  important  ones  to  know  except  in  cases  where 
it  is  expected  that  a  long  time  might  be  spent  in  the  transient 
states  before  entering  the  set  of  ergodic  ones,  thus,  making  the 
optimal  decisions  there  more  worth  knowing. 

Some  possible  applications  for  these  models  now  follow  in 
Section  4. 


4.  Applications 

In  this  section  some  possible  applications  of  coherent 
system  repair  models  or  some  variations  of  them  are  looked  at. 
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Consider  a  multiitem  inventory  model.  The  inventory  manager 
is  seeking  to  control  inventories  of  n  products  over  an  infinite 
continuous  time  horizon  so  as  to  minimize  either  his  long  run 
expected  cost  per  unit  time  or  his  total  expected  costs.  The 
demand  for  product  i  is  a  random  variable  with  exponential 

-*±t 

distribution  =  1  -  e  =  probability  {product  i  is  demanded 
before  time  t}.  The  continuous  probability  distribution  implies 
that  only  one  unit  will  ever  be  demanded  at  a  single  time.  Suppose 
that  each  time  an  item  is  demanded  (and  sold  if  available),  the 
manager  (who  always  knows  the  stock  of  various  items)  can  decide 
to  either  do  nothing  about  product  i  for  any  i  below  initial 
stock  level,  or  to  order  it  u£  to  the  initial  stock  level.  If  a 
demanded  item  is  not  available  in  stock  then  a  penalty  cost  "p" 
is  incurred,  the  demand  is  lost,  and  the  item  must  be  ordered  up 
to  the  initial  stock  level.  There  is  a  cost  K^/unit  to  order  the 
i^  product  and  a  fixed  charge  L  j>  0  per  order  placed.  If 
orders  are  assumed  to  arrive  immediately,  then  this  inventory  model 
is  just  the  Degradation  Model  for  a  series  system  (Section  4.2) 
with  parameters: 

1)  n  =  number  of  products  keeping  track  of  =  number  of 

components 

2)  =  space  allotted  in  warehouse  for  storing  item  i 

(in  terras  of  number  of  items  capacity)  =  number 
of  degradation  states  for  i 
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3)  vi ^  =  mean  time  between  demands  for  i  =  mean  component 
lifetimes  (independent  of  degradation  state). 

I 

4)  =  l  •  =  cost  to  order  l  items  =  cost  to  repair 

i  in  degradation  state  St. 

5)  p  =  penalty  cost  for  unsatisfied  demand  of  an  item  = 
system  failure  cost. 

6)  L  =  fixed  cost  to  order  =  fixed  cost  to  repair. 

If  L  =  0,  using  the  theorems  of  Section  4.2  for  V 
optimality,  the' results  are  quite  interesting.  The  V  optimal 
decision  on  component  i  is  to  repair  it  whenever  its  degradation 
state  gets  up  to 


.  Ki  J‘Ki 

^argmrn  —  -  — 

L  V4 


for  all  j 


1=0 


Thus,  every  policy  is  V optimal  and  to  find  the  optimal  ordering 
policy,  Vq  results  for  the  series  degradation  model  for  the  special 
case  described  above  would  have  to  be  studied.  For  the  maintenance 
model  application,  the  V  ^  solution  was  sufficient  because  of 
flexibility  in  changing  cost  or  other  parameter  data  at  some 
insignificant  decimal  point  to  break  "ties"  in 


K 


j 

i 
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This  procedure  is  clearly  not  allowable  in  the  inventory  application. 
The  important  point,  however,  is  the  fact  that  certain  inventory 
problems  can  be  formulated  and  solved  as  coherent  system  repair 
models  and  vice-versa. 

Although  useful  in  solving  certain  specialized  inventory 
problems,  the  most  useful  applications  of  coherent  system  repair 
models  would  be  in  more  standard  system-maintenance  type  situations. 
These  would  include: 

(1)  maintenance  and/or  surveillance  of  complex  electronic 
and/or  mechanical  systems 

(2)  maintenance  of  the  human  body 

(3)  inspection  and  control  of  pollutants  in  the  environment 

(4)  maintenance  of  ecological  balance  in  populations  of 
of  plants  and  animals. 

Given  the  specific  model  structure  and  assumptions  of 
coherent  system  repair  models,  they  would  be  most  useful  in 
modeling  maintenance  problems  where  policy  decisions  are  to  be 
made  over  a  long  (  infinite  time)  horizon,  and  decisions  are  not 
likely  to  be  made  at  times  other  than  the  instants  of  a  component 
failure.  To  use  the  model  directly,  the  components  should  have 
approximately  an  exponential  (constant  failure  rate)  or  Erlang 
distribution.  However,  there  are  model  extensions  which  could 
approximately  handle  the  case  of  nonexponential  or  Erlang  components, 
(see  Section  5). 
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Some  specific  examples  of  possible  applications  of  various 


coherent  system  repair  models  or  extensions  of  them  are  now 
presented. 

Example  7.1  (A  Replacement  Model):  Suppose  one  has  a  complex 
electronic  or  mechanical  unit  which  consists  of  n  distinct  (some 
could  be  identical)  components  which  form  a  coherent  system  and  have 
approximately  a  constant  failure  rate.  If  failed,  a  component  i 
can  either  be  left  untouched,  as  long  as  the  system  is  still  operating 
or  replaced  at  a  cost  K^.  There  are  an  unlimited  supply  of  spares 
for  each  component.  If  the  system  fails,  there  is  a  penalty  p 
incurred.  This  is  an  example  of  a  Basic  Model  problem.  If  the 
components  can  be  in  degradation  states,  then  one  has  a  Degradation 
Model. 


Example  7.2:  Suppose  one  has  a  deteriorating  system  of 
components  for  which  maintenance  decisions  like  in  Example  7.1  can 
be  made  at  the  times  of  component  failures.  In  this  case,  however, 
the  operator/manager  is  concerned  not  so  much  with  costs,  but  with 
keeping  the  expected  fraction  of  time  the  system  will  be  failed, 
at  a  minimum.  Repair  of  components  is  no  longer  instantaneous, 
otherwise  the  system  can  obviously  be  kept  always  operating.  The 
Noninstantaneous  Repair  Model  can  be  used  to  model  this  situation 
if 
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(1)  set  l  -  K£  =  0 

(2)  set  p  =  a  ■  mean  repair  time 

(3)  assume  p  is  incurred  in  every  state  where  the 
system  is  down. 


Example  7.3:  Suppose  we  have  to  decide,  in  constructing 
a  system  of  n  identical  units  in  parallel,  how  large  to  make  n 
so  as  to  insure  that  the  system  will  be  operating  on  the  average 
at  least  95%  of  the  time,  if  system  is  to  be  maintained  by  a  single 
server  who  has  a  mean  service  time  of  a)  .2,  b)  .3,  or  c)  1 
unit  of  time.  The  mean  life  of  a  component  is  1  unit  of  time  and 
failure  rate  is  constant. 

Using  the  results  of  Section  5.2,  run  the  model  for 
a  =  .2,  .3,  1.0  for  increasing  values  of  n  in  a  parallel  system 
until  minimum  V_^  is  _<  .05.  Use  *  0,  p  =  o,  £  ■  0. 


q  =  .  2  (policy  ^  does  it) 


n 

2 

3 

min  V  , 

.06 

.025 

-1 

n 

2 

3 

4 

min  V  ^ 

.101 

.062 

.047 

n  =  3  does  it. 


n  ■  4  does  it. 
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0  =  1 


n 

2 

3 

4 

min  V_1 

.4 

.375 

.37 

impossible  for  any 
n,  can  not  use  such 
a  slow  server! 


This  same  type  analysis  could  be  done  for  very  complicated 
systems  with  components  having  Erlang  lifetimes  or  degradation 
states. 


Example  7.4:  Suppose  an  electric  power  company  is  trying  to 
maintain  n  -  load  generators,  which  form  an  electrical  system 
network.  The  system  is  working  if  and  only  if  electrical  demand 
is  satisfied  for  every  customer  in  the  region  covered.  The 
generators  can  be  observed  in  a  number  of  degradation  states  and 
repair  is  noninstantaneous.  A  combination  of  Noninstantaneous  Repair 
and  Degradation  Models  can  be  used  to  model  this  situation.  The 
solution  would  have  to  be  found  on  the  computer.  Even  had  the  model 
been  simple  enough  to  apply  some  of  the  results  of  this  thesis,  they 
would  not  apply,  because  the  system  defined  here  is  noncoherent .  The 
nature  of  load  generators  is  such  that  demand  might  not  be  satisfied 
given  a  certain  subset  working  at  a  certain  degradation  level,  but 
it  might  be  satisfied  with  some  subset  of  those  generators  working. 
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Example  7.5  Fault  Tolerant  Systems  [24]:  Fault  tolerant 
computing  is  a  rigorous  discipline  covering  the  design,  analysis 
and  maintenance  of  highly  reliable  computer  systems.  Having 
started  in  the  1960’s  due  to  the  space  programs,  the  subject  now 
embraces  a  wide  spectrum  of  problems  concerning  the  reliability 
of  computer  systems.  A  Markov  model  for  evaluating  the  reliability 
of  such  systems  has  been  established  in  [24].  It  assumes  the  fault 
tolerant  computer  to  be  made  up  of  n  independent  modules  which 
can  each  have  a  given  number  of  spares.  This  is  the  parallel 
subsystem  in  series  setup  mentioned  in  Example  6.1.  Given  a  level 
of  spare  redundancy,  the  Markov  process  is  assumed  to  start  with 
all  components  working  and  it  runs  until  everything  fails.  The 
system  is  assumed  to  be  nonrepalrable  or  closed,  thus,  all  the 
system-failed  states  can  be  lumped  into  a  single  state.  The  system 
reliability  at  any  time  t  is  then  just  the  sum  of  the  probabilities 
of  being  in  the  working  states  at  time  t  given  that  one  started 
with  everything  working. 

One  new  concept  utilized  by  Ng  in  [24]  is  the  idea  of 
coverage.  See  Arnold  [1]  or  Bouricius  et  al  [4]  for  a  discussion 
and  its  effects.  The  coverage  of  a  component  is  the  probability  that 
if  it  fails  and  there  are  spares  available,  that  the  system  will 
detect  the  failure  and  switch  in  a  spare.  Perfect  coverage  (=  1) 
has  been  assumed  throughout  in  coherent  system  repair  models. 
Imperfect  coverage,  even  .98  or  .99  has  been  shown  in  [4]  to 
significantly  effect  system  reliability.  With  minor  modifications 
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to  the  state  space,  imperfect  coverage  can  be  introduced  and  incorporated 
into  coherent  system  repair  models.  The  process  then  becomes  a 

•f 

semi-Markov  process  because  the  transition  probabilities  now  depend 
on  which  unit  has  failed  (thus,  the  next  state)  as  well  as  on  the 
current  state.  Veinott's  solution  technique  still  works  in  this 
case  (see  Denardo  [11]). 

Examples  7.1  through  7.5  have  given  some  indication  of 
possible  uses  and  applications  of  coherent  system  repair  models. 

There  are  many  possible  real-world  maintenance  problems  which  could  be 
formulated  as  such  a  model,  but  given  one,  it  is  unlikely  that  its 
system  parameters  would  be  such  that  one  of  the  theorems  would  apply 
directly  to  solve  the  problem.  In  such  cases,  optimal  solutions 
would  be  found  computationally. 


5.  Extensions  and  Topics  for  Future  Research 

In  this,  the  final  section,  some  possible  topics  for  future 
research  in  the  coherent  system  repair  model  area  are  covered.  These 


fall  into  three  categories: 


(1)  Additional  theoretical  results  for  models  falling 
within  the  given  coherent  system  repair  model  structure. 

(2)  Extensions  of  coherent  system  repair  model  structure  and 
possible  theoretical  results  forthcoming. 
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(3)  Computational  techniques  for  obtaining  solutions. 


Probably  the  most  important  topics  to  be  worked  on  at  this 
point  are  the  computational  techniques.  There  were  lots  of 
theoretical  results  obtainable  for  the  simplest  coherent  system 
repair  model,  the  Basic  Model.  The  Degradation  and  Noninstantaneous 
Repair  Models,  being  more  complex,  yielded  fewer  theoretical  results. 
Clearly,  to  get  many  more  results  for  the  previously  treated  models 
or  extensions  of  them,  computational  techniques  will  be  necessary. 

The  method  in  Chapter  VI  using  MINOS  is  okay  for  small  to 
moderate  sized  problems  (up  to  10-15  components  for  a  general  system) 
For  any  large  sized  Basic,  Degradation,  or  Noninstantaneous  Repair 
Model  or  any  more  complex  extension  of  these,  more  efficient 
algorithms  will  be  needed  as  well  as  more  theory  which  would  allow 
one  to  restrict  the  decision  or  state  space  initially  before  going 
to  the  computer.  Extensions  of  Theorem  6.3  on  the  independent 
series  subsystem  technique  to  other  cases  besides  V  ^  and  a 
Basic  Model  would  be  most  desirable. 

I  would  expect  that  the  development  of  an  efficient  code 
implementing  the  policy  improvement  algorithm  for  finding  minimum 
V_^,  Vq  might  do  the  job  better  than  a  linear  program  would  for 
the  same  size  problem.  This  would  allow  one  to  solve  problems 
which  have  countable  state  spaces  such  as  the  time-dependent 
extension  of  the  Basic  Model  to  be  introduced  later  in  this  section. 
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Once  an  efficient  computational  technique  has  been  set  up 
for  solving  at  least  moderate  sized  problems,  then  not  only  will 
the  possibility  of  finding  the  ODtimal  solutions  to  more  complex 
real-world  problems  exist,  but  also  the  possibility  of  observing 
solution  behaviors  which  would  lead  to  formulation  of  more  theorems 
restricting  the  policy  space.  Then,  perhaps  even  larger  problems 
could  be  solved. 

More  immediately,  with  the  assistance  of  the  MINOS  LP- 
procedure  outlined  in  Chapter  VI,  the  optimal  solution  structures, 
both  V_^  and  Vq  need  to  be  more  thoroughly  studied  for  the 
Degradation  and  Nonlnstantaneous  Repair  Models  or  combinations  of 
the  two.  Open  problem  gaps  in  Tables  7.7  and  7.8  need  to  be 
resolved  concerning  cases  when  it  is  never  optimal  to  repair  when 
more  than  k-units  are  up  in  a  k-of-n  system  or  never  optimal  to 
repair  more  than  one  unit  at  a  time  in  an  ergodic  state.  Due  to 
complexity  of  calculations,  no  results  have  been  yet  obtained 

for  either  model  and  very  little  for  V  in  the  case  of  more  than 
one  server  or  nonidentical  components.  Some  examples  of  possible 
problems  to  be  studied  in  these  areas  are: 

(i)  If  repair  is  instantaneous,  then  in  many  models  it  is 
never  optimal  to  repair  more  than  one  unit  at  a  time 
in  an  ergodic  state,  so,  one  server  is  all  that  is  needed. 
In  the  nonlnstantaneous  repair  case,  two  examples  were 
given  where  this  is  false.  However,  in  both  examples 
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even  though  more  than  one  server  was  needed,  repair  was 
never  initiated  on  more  than  one  unit  at  a  time.  Would 
the  rewording  of  statement  (2),  Section  7.2,  make  it 
true  in  the  noninstantaneous  repair  case? 

(ii)  If  p  is  small  enough  in  a  k-of-n  system,  then  one 

never  repairs  up  to  more  than  k  +  1  components  working 
from  any  transient  state.  Is  this  true  for  larger  p 
(yes,  if  parallel,  k  =  1)  or  more  generally,  is  it  true 
that  in  a  general  coherent  system  that  one  never  repairs 
up  to  more  than  5.  +  1  components  working  where  1  is 
the  number  of  components  in  the  largest  minimum  path  set? 

As  far  as  possible  model  extensions  go,  there  are  many  possible. 
The  Markov  decision  chain  formulation  is  very  general  and  many  of  the 
maintenance  models  for  deteriorating  units  in  categories  mentioned  in 
Chapter  I,  Section  2,  can  be  extended  to  include  systems  of  n 
independent  components  as  with  coherent  system  repair  models.  Of 
course,  some  would  probably  have  too  many  states  to  be  solvable,  even 
on  the  computer,  not  being  of  much  use  in  that  case.  Two  possible 
extensions  which,  although  are  quite  complicated  statewise,  would  be 
very  important  from  the  standpoint  of  applications  to  be  able  to 
formulate  and  solve,  would  be  a  time-dependent  model  and  an  uncertain 
information  model. 

There  are  many  maintenance  situations  where  failure  rates  of 
components  are  not  constant  and  a  time  factor  needs  to  be  accounted  for. 
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This  can  be  incorporated  into  the  coherent  system  repair  model 
structure  if  one  assumes  that  all  component  lifetimes  and  service 
times  are  integer  valued,  (although  one  could  take  on  any 
distribution  with  this  structure).  This  keeps  the  state  space 
countable  or  finite.  A  simple  example: 

Example  7.6:  Basic  Model  n  =  2,  parallel  system. 

Time  Dependent  Extension 

=  life  of  component  number  1  =  1,  probability  1/2 

2,  probability  1/2 

L2  =  life  of  component  number  2  =  1,  probability  1/2 

3,  probability  1/2 

The  problem  can  be  formulated  as  a  Markov  decision  chain  where  the 
state  of  the  system  is  the  "age"  of  each  component.  States  are 
similar  to  those  in  the  Degradation  Model,  but  decisions,  as  in  a 
Basic  Model,  can  be  made  to  repair  any  subset  of  components  which 
are  failed. 

In  our  example,  Both  Down 


Both  Components  Up _  No.l  Up  No. 2  Up  (system  failed) 


State? :  1q2q  1^2 

12  12  12 

0  0  1  Vl  T2 

1122  X0  X1 

2  2  2 

0  1  2 

0 

Decisions: 

do  nothing 

a,r2 

A,R: 

A,RrR2,R12 

Costs: 

0 

o,k2 

0,^ 

O.Kj+p, 

k2+p,k1+k2+p 

where  N.  means 

component  N  is 

at  age  i. 
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Transitions  still  occur  at  the  instants  of  component  failures, 
but  now  depend  on  the  "ages"  of  the  components  as  well  as  just 
"which"  components  are  working  and  the  decision  taken.  So, 


1o2o,t 


1M  , 
0  , 


lv  0,  22,  21 


otherwise 


t  < 

A2' 


1/2, 
0  , 


t  =  i1,  0 
otherwise 


\  ,  etc. 


If  desired,  the  decision  space  could  be  expanded  to  include 
repair  of  a  nonf ailed  but  "aged"  component.  Also,  the  distributions 
of  and  L 2  can  be  adjusted  to  be  IFR,  DFR  or  whatever  failure 

rate  assumption  is  desired. 

In  the  uncertain  information  area,  the  model  of  Rosenfield  [26] 
could  be  applied  directly  and  modified  to  the  case  of  a  coherent  system 
repair  model.  Components  can  either  be  assumed  to  have  exponential  or 
discrete  lifetimes.  The  state  of  the  system  can  be  represented  as 
(s,  t)  where  s  =  working  configuration  of  components  last  known  with 
certainty  and  t  =  time  units  since  perfect  information. 

Decisions  available  in  a  state  (s,  t)  are: 

A)  do  nothing 

B)  repair  some  subset  of  known  failed  components  in  s  at 
usual  coherent  system  repair  model  costs. 
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(C)  pay  an  inspection  cost  M  >  0  to  ascertain  exact 

state  of  system  and  then  repair  or  not,  based  on  true 
value  of  s. 

A  possible  application  of  this  extension  is  the  following: 

The  control  rods  in  a  nuclear  power  generating  system  form  a 
k-of-n  system.  The  cost  of  inspecting  is  high  but  the  cost  of 
system  failure  could  be  even  higher.  This  model  could  help 
determine  the  optimal  time  intervals  between  inspections  as 
well  as  the  optimal  decision  given  inspection  results. 

Other  topics  for  possible  future  research  on  coherent  system 
repair  models  include: 

(1)  the  introduction  of  a  discount  rate,  a,  in  which  case 
none  of  the  theory  presented  applies  but  policy 
improvement  can  be  used  to  find  an  optimal  policy  with 
little  more  work  than  required  for  the  a  =  0  case. 

(2)  changing  to  a  finite  time  horizon  in  which  case  neither 
the  theory  nor  the  computational  methods  presented  in 
this  thesis  apply.  The  types  of  results  obtainable  here 
would  have  to  be  computational  or  theoretical  but  with 
limited  prospects  of  obtaining  useful  results  compared 
to  the  infinite  horizon  case.  The  following  is  a 


discrete  time  example: 


-wrw 


Example  7.7:  Consider  a  single-item  inventory  model  whose 
time  horizon  is  finite  over  n  periods.  The  objective  is  to 
minimize  the  total  expected  n-period  costs.  Observe:  stock  at 
the  beginning  of  each  period  (x^)  and  then  order  a  nonnegative 
amount  up  to  the  storeroom  limit  N  -  1  for  instantaneous  delivery 
to  get  the  starting  stock  y^  at  the  beginning  of  period  i.  The 
demand  occurs  according  to  the  distribution 


$^(jjy  )  =  initial  stock  xi+1  in  period  i  +  1  given 


starting  stock  y^  in  period  i 


Cost  =  c^(z)  of  ordering  z  units  =  z  •  K 


1, 

0,  otherwise 


"*i-l 


Storage  cost,  G^y  )  =  G(yi>  = 


P,  y±  =  0 
0,  otherwise 


Demand  is  "backlogged"  and  if  a  shortage  of  an  item  occurs,  ordering 
is  required.  This  is  the  finite  horizon  discrete  time  version  of  a 
parallel-system,  N-identical  component  Basic  Model  so  results 
obtained  by  Veinott  [32]  for  the  single  item  inventory  model  apply 
to  this  case. 

Let  f^(x)  =  minimum  expected  cost  in  periods  1,  ...,  n 

given  x  =  initial  inventory  in  period  i.  Under  suitable  regularity 

conditions,  f  (x)  =  min  (c^y-x)  +  G^(y)  +  E[f^^(Xj,+^)  |y]>. 

1  N-l_>y>xv  0 

vi  -  1,  ...,  n  and  f  .,(•)=  0  where  y  =  y, (x)  >  x  is  the  starting 

n+i  i 

stock  in  period  i  and  xvO  =  max{x,  0}. 
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Two  results  are  obtained: 


A) 

If  ci(-) 

is  convex 

,  then  y ^ (x) 

t  X 

B) 

If  Gi(-) 

is  convex 

,  unsatisfied 

demands  are  back' 

logged,  and 

demands 

independent , 

then 

x  -  yi(x)  t 

x,  i. e . , 

y^(x)  does 

not  increase  as 

fast  as  x. 

These  results,  which  hold  in  this  case,  follow  from  Veinott's  theory 

of  minimizing  subadditive  functions  on  a  sublattice.  [See  [32]]. 

To  prove  these  results,  a  lot  of  work  (by  Veinott)  was 

necessary  but  yields  are  poor  from  a  practical  maintenance  standpoint. 

* 

Result  A)  states  that  for  any  time  period  i,  if  6^(k)  =  =  optimal 

decision  in  period  i,  k  units  working,  R^  denoting  repair  of  l  units, 

then  if  k'  >  k,  6^(k')  =  R^,,  £'  >_  fc  -  k'  [cannot  repair  more  than 

one  less  in  state  k  +  1  than  was  repaired  in  state  k] . 

Result  B)  states  that,  however  many  you  repair  in  state  k, 
you  cannot  get  a  policy  to  repair  more  than  that  in  any  state  k'  >  k 
in  any  one  time  period  and  still  be  optimal.  This  result,  which 
somewhat  limits  the  possible  set  of  optimal  policies,  is  weak  when 
one  considers  that  in  the  stationary  infinite  horizon  case,  an  exact 
optimal  policy  is  easily  found.  Of  course,  considering  that  this 
is  a  time-dependent  model,  which  could  prove  to  be  much  more  difficult 
to  solve  even  for  infitite  horizon  problems,  it  may  be  significant. 

Given  a  continuous  time  coherent  system  repair  problem  with  finite 
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planning  horizon  T,  similar  techniques  -  using  ft(s)  and  the 
model  structure  to  define  a  recursion  relation  and  then  somehow 
proving  some  results  using  the  relation  so  obtained  -  should  be 
employed. 


Example  7.8:  Parallel  system,  Basic  Model.  Finite 
horizon,  T. 

Let  ft(s)  =  minimum  expected  cost  given  you  are  in  state  s 
with  "t"  amount  of  time  to  go. 

Let  6t(s)  =  optimal  policy  decision  in  state  s  with  t 
units  of  time  to  go. 

Denote  R^  as  the  decision  to  repair  the  subset  ft  of  failed 
ft  s 

s 

components  in  state  s.  Also  R  =  A 


ft(s)  =  min  ft  (s) 


where 


Rft 

ft  S(s) 


~x  l  Xi 

T-t  /  \  iGsUft 

l  (..  L  )  •  8  dT 


0  liSsUft 


+  l  K  +  L(+p,  if  s  =  0) 

ieft  x 


This  whole  thesis  has  considered  three  variations  within 


the  coherent  system  repair  model  structure  presented  in  Chapter  I. 


Although  each  gave  different  optimal  policies  and  involved  different 


f  ! 

j 

* 

i 

-  1 

theorems  to  get  them,  all  were  formulated  as  a  continuous  time  j 

Markov  Decision  Model  with  a  finite  state  and  decision  space.  Results 
were  always  based  on  minimizing  long  run  expected  cost  per  unit  time 
or  total  expected  cost  in  the  case  of  V  ^  ties,  the  solution 
technique  being  to  enumerate  the  policies  and  either  find  an  optimal 
one  or  to  eliminate  certain  ones  from  being  optimal  given  the  specific 
Markov  transition  probabilities  and  costs  for  the  given  model. 

The  key  assumption  on  the  system  which  allowed  Markov  chain 
formulation  was  that  of  exponential  component  lifetimes  and  repair 
times  (if  non- instantaneous) .  This  is  what  guaranteed  the  Markov 
property,  that  for  every  state  of  the  system  (which  is  a  function  of 
the  component  states),  the  probability  distribution  of  the  length  of 
time  spent  in  that  state  was  independent  of  what  states  the  system 
had  been  in  previously.  By  breaking  up  component  lifetimes  or  repair 
times  into  stages,  we  were  able  to  extend  the  possible  distributions 
to  Erlang,  while  keeping  the  Markov  property.  However,  for  other 
distributions,  the  Markovian  property  is  lost  and  the  Markov  decision 
chain  approach  is  no  longer  applicable,  as  can  be  seen  in  the 
following  simple  example: 
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So , 

~  F1(t),  mean 
L2  ~  F2(t) ,  mean  p2  . 

Suppose  we  define  the  states  as  before,  being  the  configuration 
of  working  components.  Start  with  both  components  new,  state  12. 

It  is  easy  to  compute  the  holding  time  distribution  in  state  12 
as  well  as  the  probability  of  going  to  states  1  or  2  (depending 
on  which  component  fails  first).  However,  then  problems  start. 

The  holding  time  in  state  1  is  just  the  distribution  of 
given  it  has  lasted  through  the  first  transition,  a  quantity  which 
depends  on  how  long  component  1  has  been  up  for,  information  which 
is  not  kept  track  of  by  the  simplest  time-free  definition  of  states. 
To  have  time-dependent  states  would  require  an  uncountably  infinite 
number  of  states  given  continuous  component  lifetimes,  getting  into 
the  realm  of  diffusion  processes.  If  component  lifetimes  had  a 
discrete  distribution,  then  we  have  the  time-dependent  model  of 
Example  7.6. 

It  would  be  useful  to  develop  some  kind  of  a  model  which 
allows  for  general  component  lifetime  distributions  while  still 
retaining  the  same  state  and  decision  spaces  defined  in  the  Basic 
Model.  This  would  be  a  very  interesting  but  probably  difficult 
problem. 


*»»*«*.-* 
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by  Randall  E.  Fleming 

Four  types  of  costs  can  be  incurred:  component  replacement  charge, 
penalty  cost  for  system  breakdown,  fixed  charge  for  repair,  and  per-hour  labor 
charge.  Decisions  to  repair  some  subset  of  failed  components  or  to  do  nothing 
are  made  immediately  following  a  component  failure.  If  the  system  fails,  some 
repair  must  be  performed.  A  policy  consists  of  specification  of  a  decision  in 
every  state  of  the  Markov  chain  accessible  from  the  assumed  initial  state  of 
all  components  new.  Structures  of  and  restrictions  on  optimal  policies  are 
studied. 

Three  types  of  coherent  system  repair  models  are  treated:  (I)  a  Basic 
Model  in  which  components  are  either  working  or  failed,  (II)  a  Degradation 
Model  in  which  working  components  can  degrade  before  failure,  and  (III)  a 
Noninstantaneous  Repair  Model  in  which  repair  times  are  exponential,  there 
being  a  finite  number  of  servers.  The  first  two  models  assume  instantaneous 
repair  and  unlimited  service. 

Two  types  of  results  are  obtained  for  these  models  -  ones  which  obtain 
a  unique  optimal  policy  for  a  special  case  and  ones  which  restrict  the  number 
of  possible  optimal  policies  for  a  more  general  case.  Two  new  policy  types 
are  encountered  in  the  case  of  no  fixed  charge.  These  are  ones  which  "never 
repair  until  a  certain  number  of  components  are  left  working”  and  ones  which 
"never  repair  up  to  more  than  a  certain  number  of  components  at  once"  or  "never 
repair  more  than  one  component  simultaneously  in  an  ergodic  state".  The  first 
type  of  policy  appears  in  k-of-n  systems  with  Instantaneous  Repair  Models. 
Precise  optimal  policies  are  computed  for  the  series  degradation  model  and 
numerous  other  cases  in  which  the  components  are  identical.  Optimal  policy 
restriction  theorems  and  optimal  policy  computations  are  based  on  policy 
enumeration  with  comparisons  between  long  run  expected  costs,  and  in  the  case 
of  ties,  total  expected  costs. 

Computational  procedures  using  linear  programming  and  policy  improvement 
routines  are  discussed  and  an  LP  routine  is  implemented  on  some  simple  test 
cases.  Effects  on  and  sensitivity  of  the  minimum  cost  values  and  optimal 
policies  of  the  models  to  various  component  and  system  parameters  is  discussed. 
The  paper  concludes  with  a  look  at  the  numerous  possible  applications  and 
related  topics  for  future  research. 
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